As we know that the magnitude of the resultant of two vectors $X$ and $Y$,
$R^{2}=X^{2}+Y^{2}+2 X Y \cos \theta$...(i)
where, $\theta$ is the angle between $X$ and $Y$. Putting, $X=(A+B)$
$Y=(A-B)$
and $R=\sqrt{A^{2}+B^{2}}$ in Eq. (i), we get
$A^{2}+B^{2}=(A+B)^{2}+(A-B)^{2}+2(A+B)(A-B) \cos \theta$
$\Rightarrow A^{2}+B^{2}=A^{2}+B^{2}+2 A B+A^{2}$
$ +B^{2}-2 A B+2\left(A^{2}-B^{2}\right) \cos \theta$
$\Rightarrow \frac{-\left(A^{2}+B^{2}\right)}{2\left(A^{2}-B^{2}\right)}=\cos \theta$
we get, $\theta=\cos ^{-1}\left[-\frac{\left(A^{2}+B^{2}\right)}{2\left(A^{2}-B^{2}\right)}\right]$