Saving $1 million or more for retirement may seem like an intimidating goal, but the process doesn't have to feel overwhelming.

While there's no one-size-fits-all method of getting started, one of the worst things you can do is nothing, says Anne Lester, a retirement expert and author of "Your Best Financial Life: Save Smart Now for the Future You Want," which will be released in March.

"Give up this notion that there's a perfect answer," she tells CNBC Make It. "There's pretty good to terrible. Pretty good is saving 15% of your money for retirement while terrible is not saving at all."

But don't panic if saving 15% of your annual income is a bit out of reach right now. It's OK to start small.

In fact, you could start by setting aside $20 a day, five days a week.

Say you begin doing this at age 25. By putting what amounts to $100 a week into a retirement investment account that generates a 7% annual rate of return, you'd have about $1,057,681 saved up by the time you reach age 65, according to CNBC's calculations.

In this example, you're setting aside $400 a month. But thinking about it as $20 a day can feel a lot less daunting when you're looking for money to divert toward your retirement savings.

"Whenever you have a big goal, you don't do it by starting and finishing, right?" Lester says. "You break it up into small, digestible pieces."

With that in mind, CNBC calculated how a $100 weekly contribution to a retirement investment account could grow exponentially over time if you started at ages 25, 30 and 35. These calculations assume the account generates a 7% annual rate of return and don't account for unpredictable factors such as market volatility or periods of unemployment.

The results: If you started saving $100 a week at age 25, you'd have over $1 million by age 65. If you start at age 30, your retirement savings would have grown to around $726,000 by 65. And if you began contributing $100 a week when you turned 35, you'd have close to $500,000 by retirement.

As the chart above demonstrates, when you start early, you give your contributions a longer amount of time to grow through the power of compound interest. Using this example, waiting just five years to begin saving at age 30 instead of 25 results in a difference of more than $330,000.

Saving for retirement doesn't mean depriving yourself

Finding a small amount of money in your budget to put toward your retirement goals doesn't necessarily mean giving up little luxuries like lattes or dining out with your friends.

"I'm not going to 'latte shame' anyone. If it's a big, incredibly powerful part of your life then go buy the latte, just budget for it," Lester says.

It's important to take a look at your finances and get a clear idea of how you're spending your money, she says. That way, you can accurately figure out if you're spending money on things that may not be important to you. You may find that you're paying for a subscription you're not actually using and be able to save that money for retirement instead.

And the sooner you start, the better. When it comes to saving for retirement, "time is the single most important part of the investing equation," Lester says.

"The earlier you can start, even if it's small, the less you will actually have to save to hit that same goal in your 60s," she says. "It may be a sacrifice now, but it's saving you from a much bigger sacrifice later."

However, it's not the worst thing in the world if you can't afford to save for retirement until you're in your 30s or later, Lester says. 

If that's the case, it may mean you'll need to make larger contributions in order to meet your retirement goals since your money will have less time to compound. But be aware of the latest retirement contribution limits.

Ultimately, no matter what age you are, getting started is crucial, Lester says.

"It's about taking small steps when you set this huge goal," she says. "It's a goal to get to, but you don't have to get there tomorrow."

DON'T MISS: Want to be smarter and more successful with your money, work & life? Sign up for our new newsletter!

Get CNBC's free Warren Buffett Guide to Investing, which distills the billionaire's No. 1 best piece of advice for regular investors, do's and don'ts, and three key investing principles into a clear and simple guidebook.

CHECK OUT: Want to retire at 65 with $2 million? Here’s how much to save each month