It's impossible to remember every single thing, but how do we know if our forgetfulness is harmless or a sign of serious brain issues on the horizon?

Dr. Richard Restak has written more than 20 books about the brain. In "The Complete Guide to Memory: The Science of Strengthening Your Mind," he dives into normal memory lapses versus serious memory concerns.

Let's say "you drive to a shopping center to get a particular item, and you go in to buy the item. You come out and you can't remember where you parked your car," says Restak, a clinical professor of neurology at The George Washington University School of Medicine and Health Sciences.

"I don't think that's anything to worry about. People complain about that all the time."

But if you "couldn't remember whether you came by car or whether you took a bus or somebody dropped you off, that would be abnormal. That would be something you would seek a doctor's help [for]."

Is my forgetfulness normal or should I be concerned?

In this book, Restak offers up these indicators of what he says are normal memory lapses and potential causes for concern:

Normal memory lapses

• Remembering childhood moments differently than your siblings
• Taking the wrong exit on the way home
• Forgetting the names of people you just met or have only seen a few times
• Needing a shopping list for groceries
• Forgetting where you parked your car

Potential signs of serious memory concerns

• Forgetting how to play games you've played often and are really good at, and not remembering what's already happened in the game as you're playing
• Not being sure how you arrived at a place (by car or bus, driving yourself or being driven by someone else)

Many of the memory lapses people worry about are actually fairly common, Restak says. But, you should still challenge yourself by engaging in exercises to sharpen the muscle.

Restak himself completes a memory challenge each morning. He writes down 10 words before leaving the house to walk his dog and recites them from memory when he returns.

"Avoid boredom, stay challenged," he says.

Want to make extra money outside of your day job? Sign up for CNBC's online course How to Earn Passive Income Online to learn about common passive income streams, tips to get started and real-life success stories.

Plus, sign up for CNBC Make It's newsletter to get tips and tricks for success at work, with money and in life.