Well, maybe that’s not 100% true – your definition and mine are likely somewhat different. In the backup and disaster Recovery (DR) world, many hear the terms RPO and RTO in the context of recovering from a catastrophic outage (building burns down, hard drive or SAN crashes, etc.). What exactly do these terms RPO and RTO mean and are there other examples of when they might come into play?
RPO – Recovery Point Objective – Following a disaster or the failure of an application and/or system, the amount of data loss (in time – for example 5 minutes back, 20 minutes back, etc.) that you can accept when services to that application have been restored. It is essentially the “Point in Time” to which the application and its data can be recovered. It is not uncommon for users to desire an RPO of “0” or “near real-time” for their most critical applications.
RTO – Recovery Time Objective – The amount of time that can elapse before a failed application and/or system is returned to service with the data matching your RPO requirements. For example, you may say that e-mail can have an RPO of 60 Minutes (you’re willing to lose the last 60 minutes of data) but it must be online and available to users in under 30 minutes (30-Minute RTO).
RPO is somewhat unique in that there is almost always data loss following a catastrophic failure. Does your company have the manual processes in place to recreate the transactions that may have been lost? More importantly, what’s the cost associated with losing data or the ability to access it, over time? Let’s use an example.
A law firm with 25 attorneys, each with an average billing rate of $300/hr. (excluding paralegals, secretaries, etc.) with no access to their documents or e-mail (contracts, correspondence, case materials, etc.) will lose at a minimum, $7,500 in revenue during a single one-hour outage. There may additionally be penalties for missed deadlines, court appearances, contract stipulations, etc. Reputation harm should be factored in as well. That same outage, lasting for just one business day – represents $60,000 in lost revenue. And we know that most attorneys work more than 8 hours per day. What if you have 50 attorneys? 100? The numbers start to become staggering.
In truth, the building did not go away, there was no fire and brimstone, the Mayan calendar shows nothing for today, not even a loss of power. All of the users are calmly sitting at their desks, waiting for the systems to come back online. However, the disaster this time is being caused by malware known as a “crypto-virus”. One of those users opened an email from a client, it looked legitimate – possibly even related to a normal day-to-day interaction between colleagues. In the background, that little email has done its dirty work. It has encrypted all of the files to which that user has access to “modify” rights. The result? A little over 2 million files, dating back 12 years, have been locked down.
How hard can it be to get those back? Well, if you ran backups hourly and the threat was caught quickly (usually, the signs start to show 3 or 4 hours into the spread), you’ll need to go back and find the recovery point just prior to when the damage occurred. Since this threat does not change the time or date-stamp, it might take several hours to determine which items were affected. If you’re not sure (and you won’t be), you’ll have to play it safe and go back to a recovery point which you know is clean. Perhaps yesterday? So now, we’ve lost last night’s work, and most of today. The costs for this small-ish firm? Let’s assume that the recovery effort will take another 8 to 16 hours. So we’re out $120k in gross revenue. But it’s really much higher. We might have to pay the consultants (and these are specialists) at least $200/hr. to resolve the threat (4 hours), and then recover your information (another 8-16 hours). So let’s just round up to be safe and say this event cost $125,000.00.
Now, you determine that you can’t deal with the loss of information (you have a “huge” multi-million-dollar case pending and some key data that only you had, was lost), despite the hard costs you’ve already incurred… You decide to pay the ransom which you can only transact via BitCoin currency (www.bitcoin.org). The ransom is cheap; only $1,000 to get a key which returns your documents to normal. Why didn’t we just do that to start?
Well, since this “crypto-currency” is not well known or trusted in the banking industry, it takes a long time to convert US dollars into BitCoins. In most cases, 7-10 days with be the minimum (so now that’s 10 days x 8 hours per day x $7,500 per hour – you get the drill) … But guess what? It took you too long to fund your BitCoin account and now the ransom has gone up, substantially! The cost is now $5,000 USD and it will take another 7 days to get those funds into your account.
What’s the moral of this story? Know what it costs to lose business (hours, customers, widgets, wages) and adjust your RPO and backup strategy to minimize the potential losses. Keep multiple copies of your backups in different locations. Don’t rely on “replication” tools to protect your information – once a file in encrypted, it simply replicates like it always did. Take an extremely aggressive approach to leveraging the tools at your disposal – firewalls, content blocking, antivirus, backups, etc. Most importantly – educate your users! 95% of these threats come in through the front door (email). Train them to be suspicious, to know how to “hover” over a link and see that the site you’ll be taken to if you click on it, is not what you’d expect based on the “description” that you see. We can help. We have to help… I implore you to take this seriously and together, we’ll do everything in our power to protect one another.
For related information, please check out the following posts on our blog: