If 5 ,   5 r ,   5 r 2 are the lengths of the sides of a triangle, then r can not be equal to:

Given, $5, 5r, 5r^2$ are the length of sides of triangle, then we know that the sum of two sides of a triangle is more than the third side i.e.

$5+5r>5r^2 ...\left(1\right)$

$5+5r^2>5r ...\left(2\right)$

$5r+5r^2>5 ...\left(3\right)$

From $\left(1\right),$

$r^2-r-1<0$

$\left[r-\left(\frac{1+\sqrt{5}}{2}\right)\right] \left[r-\left(\frac{1-\sqrt{5}}{2}\right) \right]<0$

$r\in \left(\frac{1-\sqrt{5}}{2},\frac{1+\sqrt{5}}{2}\right) ...\left(4\right)$

from $\left(2\right),$

$r^2-r+1>0$ 

The discriminant of the quadratic is $D={\left(-1\right)}^2-4\times 1\times 1=-3$ and we know that if the discriminant of a quadratic is negative and its leading coefficient is positive then the quadratic is positive for all real numbers. 

$\Rightarrow r\in R ...\left(5\right)$

from $\left(3\right),$

$r^2+r-1>0$

$\left(r+\frac{1+\sqrt{5}}{2}\right)\left(r+\frac{1-\sqrt{5} }{2}\right)>0$

So, $r\in \left(-\infty , -\frac{1+\sqrt{5}}{2}\right)\cup \left(-\frac{1-\sqrt{5}}{2}, \infty \right) ...\left(6\right)$

Now, taking intersection of $\left(4\right), \left(5\right), \left(6\right),$ we get $r\in \left(\frac{-1+\sqrt{5}}{2}, \frac{1+\sqrt{5}}{2}\right).$

Out of the given options only $\frac{7}{4}$ is not in the interval obtained.