I am reading David L Smith's "Why We Lie" these days, and it is an interesting read. He traces the roots of deception in human social interactions, as well as the phenomenon of self-deception, to the way humans have evolved. He gives examples of deception existing at all levels in nature, and then comes to the fascinating study of self-deception, which as far as we know, exists mostly in humans (if we exclude the well-known example of ostriches burying their head in the sand).
Self-deception has many forms: denial, self-censoring, repression, willful or naive ignorance of one's unconscious drives, self-righteousness, pretension, delusion, etc.
Mr Smith considers self-deception as primarily an evolutionary adjustment so as to appear truthful in one's interactions. Deceit is punished, in animals as well as in humans. In this context, self-deception can be a an evolutionary strategy: By appearing genuine, we escape detection (we are not aware that we are lying, so the tell-tale signs of lying are absent), so we fool the lie-detector filters of others by fooling ourselves first. An example is the heart-felt "I love you" which we in retrospect can see as a temporary sexual attraction. But because we ourselves believe it so strongly, we are easily able to convince the other person and get what we want.
This is as far as the book goes.
I think there is another reason behind self-deception (which is also based on our evolution). Self-knowledge, or honesty about oneself, can be very debilitating in two ways: first, it makes us behave in a less-greedy/less-instinctually-driven ways, which in itself lessens our genes' propagation potential. As we recognize our basic nature, we can then choose to override or eliminate it. In either case, we then are no longer running after potential mates, no longer trying to gain power or money, and as such we let others get ahead in the genetic race for survival and pleasure.
The second way in which a lack of self-deception can hurt is this: self-knowledge, or honesty about oneself, can lead to depression or suicide if one is not intelligent enough to recognize that the traits one has are the ones one was born with, and that those traits are the end result of evolution. That one need not blame oneself or anyone else for one's physical and mental constitution at birth (some exceptions comes to mind: e.g. a drinking mother which by her alcohol intake causes some congenital defect in the child, or a lazy father who does not feed the mother enough so as to produce a weak malnourished offspring).
Suppose I have warts on my face and have other characteristics which are considered "ugly". Suppose I admit to myself that I am in fact ugly and unattractive. This admission can lead to severe depression and suicide, if a compensatory mechanism (e.g. of cosmetic surgery, "inner beauty", intellectual accomplishment, etc.) or a supporting community (which downplays or ignores my ugliness) is absent.
In the absence of such compensations, one usually refuses to admit that "I am ugly". Similarly for a fool to clearly see that he is a fool, someone who can't do anything right, can be very depressing. Most people lie to themselves about their flaws so as to have an overall good feeling about themselves.
This good feeling, or at least the avoiding of its opposite depressive feeling, is a very important factor for most people which makes life at least somewhat tolerable, which makes one go on living and not give up, and which makes people achieve their potential in other ways.
It is, in other words, the "power of positive thinking".