The values of a and b , for which the system of equations 2 x + 3 y + 6 z = 8 x + 2 y + a z = 5 3 x + 5 y +…

The given equation is 

$2x+3y+6z=8$

$x+2y+az=5$

$3x+5y+9z=b$

So, $D=\left|\begin{array}{lll}2& 3& 6\\ 1& 2& a\\ 3& 5& 9\end{array}\right|$

$=2\left(18-5a\right)-3\left(9-3a\right)+6\left(5-6\right)$

$=36-10a-27+9a+30-36$

$=3-a$

$D_z=\left|\begin{array}{lll}2& 3& 8\\ 1& 2& 5\\ 3& 5& b\end{array}\right|$

$=2\left(2b-10\right)-3\left(b-15\right)+8\left(5-6\right)$

$=4b-20-3b+15+40-48$

$=b-13$

The system of equation has no solution if $D=0 \& D_z\ne 0$

So, If $a=3, b\ne 13$, then the system of equation has no solution.