I reckon Einstein understood a lot of things pretty well... It seems he had a pretty decent grasp on design principles too.
I'm not sure what the exact context of the quote was, but the idea is so universal that I don't think it really matters.
I believe a great logo is the simplest possible way to represent the ideals, message or story of a company through type, shape and colour.
Think about this; when you tell somebody a crucial piece of information, you try to keep things clear, simple and direct, don't you?
Well the same strategy should apply to logo design.
Keep it simple.
Humans only have limited attention spans and are constantly overloaded with information.
The best way of having your message register with them is to keep it simple.
So, to stay true to the idea at hand, I'll keep this post short.
Or as simple as I believe it can be.