Issue with shadows on custom geometry

I have an small issue with strange shadows in my custom geometries. The shadows looks jagged at some places as you can see in the images below. What causes this, wrong normals when I build my mesh? I use VertexColors.

(Zoomed in version)

Tune biases and offsets for your Light source.