Although you believe you can remember all that's worth remembering about an applicant, an interviewer's recall tends to become blurred when there are several applicants, especially when combined with the passage of even a little time. You can maintain plenty of eye contact and gather a great deal of visual data while still taking some basic notes.
As for the notion that a lack of notes will reduce the likelihood of a claim from an applicant, the best way to prevent such claims is by only asking job-related questions. One could even argue that a structured and job-related interview is a deterrent to potential lawsuits. The idea is to write down the key points that an applicant says, rather than being a court reporter.
It is interesting to note that many applicants actually appreciate having an interviewer who takes notes. This makes the applicants feel that what they are saying is of real value. It is also worth noting that when your manager asks you to follow a particular procedure during a job interview, you should do it.
| 
