f(x)=x2−mx+1 and α≤β are the roots of f(x)=0.
Now, α<β<1 implies that f(1) and the coefficients of x2 have the same sign.
This gives f(1)>0 ⇒1−m+1>0⇒m<2
Also, the discriminant is m2−4≥0.
So m≤−2 or m≥+2.
Taking intersection, we get, m≤−2.
Also, note that if m=−2, the roots are −1,−1.