Mr. Clean Magic Erasers
Mr. Clean Magic Erasers are perfect for removing all kinds of markings from almost anything except paper products. These erasers use microfine abrasive material, rather than chemicals, to sand marks away on a microscopic level. However, much to my great surprise, even though they are abrasives, they do not scar plastic unless you scrub REALLY hard for a LONG time. I used them to restore valuable old video game cartridges to like-new condition, so I didn't want the plastic to be abraded and I was amazed at how even the dirtiest, most marked up Nintendo 64 cartridge could be made to look almost brand new with Magic Erasers.
Even better, because they don't use/contain chemical cleaners, you don't have to worry about chemical odors or chemicals messing up the plastic like some chemicals can.
In addition, they are commonly found in many stores in the cleaning supplies section and they are very inexpensive, around $1.50 for a box of 2 and one eraser will last a long time.
Here is what the box looks like:
However, they sometimes have different boxes, here are some pictures.
To use them, just get them a bit damp & scrub until the marker goes away. Often the marker can go away pretty quickly. Sometimes you have to scrub for a while, but it will almost always go away after a while. Just be careful not to scrub down in the pips too much if you have paint in them, or you'll scrub off the paint.
I learned this trick from a used video game store owner when I was buying and cleaning old video games & equipment. These Magic Erasers worked SO well, getting tons of marker marks and scuff marks off video game cartridges, video game consoles, walls, etc. -- stuff I never thought would come off the plastic, and the erasers don't abrade the plastic at all, unless you scrub REALLY hard for a LONG time (then you start to see a very fine change in the way light reflects off the area).
I highly recommend them!