A 2 , 6 , 2 , B - 4 , 0 , λ , C 2 , 3 , - 1 and D 4 , 5 , 0 ,   λ ≤ 5 are the vertices…

Given:

$A\left(2,6,2\right),B\left(-4,0,\lambda \right),C\left(2,3,-1\right)D\left(4,5,0\right)$

Area of quadrilateral 

$=\frac{1}{2}\left|\overrightarrow{AC}\times \overrightarrow{BD}\right|=18$

$\Rightarrow \left|\overrightarrow{AC}\times \overrightarrow{BD}\right|36$

Now,

$\overrightarrow{AC}\times \overrightarrow{BD}=\left|\begin{array}{ccc}\hat{i}& \hat{j}& \hat{k}\\ 0& -3& -3\\ 8& 5& -\lambda \end{array}\right|$

$\Rightarrow \overrightarrow{AC}\times \overrightarrow{BD}=\left(3\lambda +15\right)\hat{i}-\left(0+24\right)\hat{j}+\left(0+24\right)\hat{k}$

$\Rightarrow \overrightarrow{AC}\times \overrightarrow{BD}=\left(3\lambda +15\right)\hat{i}-24\hat{j}+24\hat{k}$

So,

$\left|\overrightarrow{AC}\times \overrightarrow{BD}\right|=36$

$\Rightarrow \sqrt{{\left(3\lambda +15\right)}^2+{\left(24\right)}^2+{\left(24\right)}^2}=36$

$\Rightarrow 9{\lambda }^2+225+90\lambda =1296-1152$

$\Rightarrow 9{\lambda }^2+90\lambda -81=0$

$={\lambda }^2+10\lambda +9=0$

$\Rightarrow \lambda =-1,-9$

Since,

$\left|\lambda \right|\le 5\Rightarrow \lambda =-1$

Therefore,

$5-6\lambda =5-6\left(-1\right)=11$