Consider this the coloring book version of applying a black and white or "antique" filter, as you would do on your Instagram feed. Working in pencil, marker or paint, choose a neutral palette. As you can see on the top image, shading with regular pencil on your coloring book image makes it look less like a coloring book image, and more like a hand drawing. By using a palette of browns, you can create an "antiqued" look on your coloring book pages, as if they are treasures from a different era.
Embroider around the painted design. Some paints dry soft and flexible. Acrylic paint, however, dries hard. If you are stitching through the painted area, bring the needle through with caution, as making excess holes will show in the paint.