I’ve sat in hundreds, maybe thousands, of Revenue Management meetings. The setting is often the same: a promotion is launched to stimulate demand during a slow period, and a few days or weeks later, a pickup report is pulled. Someone reads off the number of rooms booked or the revenue generated, and the room collectively decides whether the tactic “worked.”
Sometimes the verdict is positive and the promotion becomes a recurring tactic. Sometimes it’s dismissed and shelved. But in most of these meetings, no one asks the most important question: compared to what?
The assumption that we can look at an outcome in isolation and know whether it was effective is one of the most quietly damaging habits in hotel pricing strategy. It’s not a matter of laziness or carelessness. It’s a matter of training. Most Revenue Managers are taught how to react to data, but not how to structure it. They’re expected to optimize in real-time but are rarely given the tools to evaluate their actions as real experiments.
If we want to truly understand the impact of a pricing tactic, we need to think like experimentalists. We need to build structures that allow us to know whether something worked, not just believe it.
What a Proper Evaluation Requires
In any field where evidence matters, evaluating the success of a tactic depends on four essential components.
First, a clear hypothesis. What outcome are you trying to influence? For which segment? Over what time period?
Second, a control group or valid baseline. What would have happened if you had done nothing?
Third, consistency of conditions. Are you comparing equivalent environments—same time frame, same audience, same distribution channel?
Fourth, a reliable outcome measure. Are you assessing the right metric in the right context?
Without these elements, you’re not conducting an evaluation. You’re drawing conclusions from noise.
Unfortunately, these fundamentals are routinely skipped in hotel Revenue Management. Here are the five most common mistakes that result.
1. No Context
One of the most frequent and fundamental errors is evaluating performance without any point of comparison. A report might show that a promotion generated $35,000 in revenue or picked up 200 room nights. That may sound like success, but without context, it is simply a number. It has no meaning on its own.
What matters is whether that revenue was above or below what would have happened without the promotion. Did it outperform forecast? Did it beat the base rate pickup from a similar prior period? Was demand elevated that week for unrelated reasons?
This kind of evaluation fails because it skips the control. Without a reference point, you cannot know if a tactic drove the outcome. You can only guess.
The solution is to build context into every performance analysis. This might mean comparing the promotion to a recent control period, using a forecasted trend, or modeling a synthetic baseline from historical booking patterns. Even a simple benchmark is better than none. What matters is that the number is not left to stand alone.
Learn More About Hotel Revenue Data Science Course
Learn More About Data Science of Hotel Revenue Forecasting Mastercourse
2. Comparing Across Channels
Another common mistake is comparing how a promotion performed across different distribution channels. For example, a Revenue Manager may note that a last-minute deal drove more bookings on OTAs than on the GDS, and conclude that OTAs are more responsive to that type of offer.
On the surface, this seems like a reasonable conclusion. But in practice, it is an invalid comparison. Each channel serves a different audience and operates under different rules. The OTA guest is not the same as the GDS guest. Their booking timelines, price sensitivity, and behavior patterns are fundamentally different.
When you compare the performance of the same promotion across two different channels, you are not isolating the impact of the tactic. You are comparing apples to oranges in two different climates, with no control for the surrounding factors.
The fix is simple in principle but requires discipline. Channel-based evaluations should be conducted within a single ecosystem. Compare the promotion to other promotions on the same platform or to prior periods within the same channel. If you want to compare responsiveness across channels, set up a matched test with consistent timing and segmentation. Anything less introduces too many confounding factors to draw a valid conclusion.
3. Comparing Across Time Periods
This is perhaps the most intuitive mistake and also one of the most damaging. A promotion that performed poorly in September is used as evidence that the same tactic should not be attempted in March. A weekend package that exceeded expectations in May becomes the model for pricing in October.
The problem is that no two calendar periods are truly alike. Seasonality, event calendars, weather patterns, and even day-of-week alignment all influence booking behavior. The conditions that shaped one period may be entirely absent in another.
Comparing results across time without adjustment violates the principle of experimental consistency. You are no longer isolating the tactic. You are introducing external variables that can completely obscure the cause of any change in performance.
The better approach is to match periods as closely as possible or normalize your comparison using demand-adjusted modeling. If that’s not feasible, design your test to include a control period during the same timeframe. At minimum, acknowledge that comparing different seasons without adjustment is not evaluation. It is storytelling.
4. Comparing Across Booking Windows
A variation of the time-based mistake is comparing results across different booking windows. For example, a promotion targeting bookings 30 days in advance may be evaluated against last-minute pickup patterns from a previous week.
This comparison is invalid because booking windows represent distinct behavioral segments. A guest booking far in advance is likely more deliberate and price-sensitive, while a last-minute booker may be driven by urgency or convenience. The way these guests respond to pricing, messaging, and availability is not the same.
If you compare their behavior without acknowledging that difference, you are not evaluating the promotion. You are misreading consumer intent.
The remedy is to segment your analysis by lead time. Only compare 30-day pickup to other 30-day pickup periods. Build booking curves for different windows and judge tactics relative to their position on that curve. If you understand your demand profile across time, you will see that behavior is not uniform—and your pricing shouldn’t be either.
Learn More About Hotel Revenue Data Science Course
Learn More About Data Science of Hotel Revenue Forecasting Mastercourse
5. Comparing Across Guest Types
Every promotion is designed with a guest in mind. Some are aimed at couples seeking a weekend escape. Others are targeted at solo business travelers or families on school holiday. Yet when it comes time to evaluate performance, many hotels ignore these distinctions and compare all promotions to one another as if they are competing in the same race.
This violates the most basic principle of experimental design: define your subject group. If you are measuring the effectiveness of a spa weekend package, you need to evaluate it within the segment it was meant to attract. Comparing it to a corporate group offer misses the point entirely.
When you judge a tactic by its performance against an unrelated audience, you will often reach the wrong conclusion. You may abandon a viable strategy simply because it didn’t outperform a completely different offer aimed at a different type of guest.
To avoid this, performance needs to be segmented not only by channel and time, but by audience. That requires better data—CRM integration, tagging in booking flows, or guest preference tracking. But without this, you are guessing at success. And guessing is not strategy.
Why Structure Matters
These five mistakes have one thing in common: they all stem from skipping the structure of an actual test. We deploy a tactic but never design an experiment. We draw conclusions but never isolate a cause. We make decisions, often with confidence, but without evidence.
This doesn’t mean Revenue Managers are doing a bad job. It means they are being asked to operate in a reactive environment without the analytical tools required to truly learn from their actions. Most RM departments are set up to move quickly, not to pause and measure.
But pausing to measure is what separates tactics from strategy. It is the difference between reacting and improving. And it begins with the humility to ask, not “Did it work?” but “Did I create the conditions to know if it worked?”
If you structure your tactics like experiments, even a failed promotion teaches you something. If you don’t, even a successful one might lead you in the wrong direction.
About the Author
Robert Hernandez is a data scientist and pricing strategist working across every sector of the hotel and restaurant industry—from limited-service chains to luxury resorts and fine dining groups. He is the founder of Ratebuckets, a hotel pricing intelligence platform that helps owners, asset managers, and GMs evaluate revenue strategies with fresh eyes and real-time clarity. His work focuses on exposing flawed assumptions in pricing logic and designing tools that bring rigor, transparency, and accountability to revenue decisions.