Except for the regular behavior specified in RFCs, every IP stack has some specific ways (due to coding policies, bugs, optimizations...) of responding to incoming traffic, especially when handling abnormal packets that do not strictly follow the RFC.
These specificities are used by network analysis software to guess what OS a remote host is running. They probe the OS by sending the host a bunch of abnormal packets (mangling parametres such as fragmentation, TCP flags, unused/reserved fields, size of packets, ...) and comparing the results with a signatures database of known operating systems.
Such tools are used by system administrators to map an heterogenous network, but also by crackers trying to learn more information about a remote host or a subnet, in order to use appropriate attacks and improve their chance of compromising the systems.
IP Personality is a netfilter module designed to be able to have different 'personalities' network wise, that is to change some characteristics of the network traffic, depending on different parameters. This especially enables fooling such tools in thinking a remote host is running a specific system when it is actually running another one, so as to hide or protect hosts that would otherwise vulnerable, or to build "honey pots".