The first and foremost safety tip is to be sharp and keep your wits about you. Be observant and avoid that which you sense to be wrong. However, crime in Sri Lanka is very low – especially against foreigners.
In a conservative, socially and economically developing milieu, it is prudent to dress sensibly, so as to avoid unwanted attention. One has to keep a lookout for touts and other wheeler-dealers, who might take you for a ride. Keep yourself informed and plan your tours well in advance, after doing your research.
In places of religious significance, follow local practices like covering your head, removing footwear before entering a shrine, etc. In case you do not wish to do all this, just opt out of going inside the sanctum sanctorum.
As far as health concerns, health standards are adequate.
While the water is chlorinated and safe to drink in cities like Colombo, unfamiliar microorganisms often cause stomach upset so bottled water is best. Be wary if you have a delicate stomach not just of the water, but washed salads, ice cubes, etc.
The CDC recommends most visitors get several shots before they go for Hepatitis A and Typhoid. Visitors should also take precautions against mosquito-borne disease like dengue fever, malaria, chikungunya fever, Japanese encephalitis and filariasis.