Let a vector a → be coplanar with vectors b → = 2 i ^ + j ^ + k ^ and c → = i ^ - j ^ + k…

Given, a vector $\overrightarrow{a}$ is coplanar with vectors $\overrightarrow{b}=2\hat{i}+\hat{j}+\hat{k}$ and $\overrightarrow{c}=\hat{i}-\hat{j}+\hat{k}$ and $\overrightarrow{a}$ is perpendicular to $\overrightarrow{d}=3\hat{i}+2\hat{j}+6\hat{k},$ and $\left|\overrightarrow{a}\right|=\sqrt{10}.$ 

If a vector $\overrightarrow{a}$ is coplanar with the vectors $\overrightarrow{b} \& \overrightarrow{c}$ then it can be expressed as the linear combination of the vectors.

Thus, $\overrightarrow{a}=\lambda \overrightarrow{b}+\mu \overrightarrow{c}$

$\Rightarrow \overrightarrow{a}=\lambda \left(2\hat{i}+\hat{j}+k\right)+\mu \left(\hat{i}-\hat{j}+k\right)$

$\Rightarrow \overrightarrow{a}=\hat{i}\left(2\lambda +\mu \right)+\hat{j}\left(\lambda -\mu \right)+\hat{k}\left(\lambda +\mu \right)$

Also, it is given that $\overrightarrow{a}$ is perpendicular to $\overrightarrow{d},$ and we know that if two vectors are perpendicular, then $\overrightarrow{a}·\overrightarrow{d}=0$

$\Rightarrow 3\left(2\lambda +\mu \right)+2\left(\lambda -\mu \right)+6\left(\lambda +\mu \right)=0$

$\Rightarrow 14\lambda +7\mu =0$

$\Rightarrow \mu =-2\lambda$

$\Rightarrow \overrightarrow{a}=\left(0\right)\hat{i}+3\lambda \hat{j}+\left(-\lambda \right)\hat{k}$

$\Rightarrow \overrightarrow{a}=\lambda \left(3\hat{j}-\hat{k}\right)$

$\Rightarrow \left|\overrightarrow{a}\right|=\sqrt{10}\left|\lambda \right|$

Given $\left|\overrightarrow{a}\right|=\sqrt{10}$

$\Rightarrow \left|\lambda \right|=1$

$\Rightarrow \lambda =1$ or $-1$

Also, if $\overrightarrow{a}, \overrightarrow{b} \& \overrightarrow{c}$ are coplanar, then $\left[\overrightarrow{a}\overrightarrow{b}\overrightarrow{c}\right]=0$

Now, $[\overrightarrow{a} \overrightarrow{b} \overrightarrow{c}]+[\overrightarrow{a} \overrightarrow{b} \overrightarrow{d}]+[\overrightarrow{a} \overrightarrow{c} \overrightarrow{d}]$

$=0+[\overrightarrow{a} \overrightarrow{b} \overrightarrow{d}]+[\overrightarrow{a} \overrightarrow{c} \overrightarrow{d}]=[\overrightarrow{a} \overrightarrow{b}+\overrightarrow{c} \overrightarrow{d}]$

On putting the values of the vectors for finding the box product, we get

$=\left|\begin{array}{rrr}0& 3\lambda & -\lambda \\ 3& 0& 2\\ 3& 2& 6\end{array}\right|$

$=-3\lambda \left(12\right)-\lambda \left(6\right)=-42\lambda =-42$ or $42.$