Most brokers and data consumers in real estate are used to asking data providers for updates. With a simple new technology model, data producers could soon be pushing out updates to consumers only when they are needed. Constellation1 has adopted RESO’s new standard in push notifications. They discussed how their efforts can improve an organization’s data-gathering efficiency during a talk at RESO 2022 Fall Conference titled “Pushing vs. polling: The Future of MLS/Broker Data Distribution” presented by Rick Herrera (Vice President of R&D Data Services at Constellation1) and chair of RESO Data Consumers Subgroup. WATCH VIDEO (27:44)
This evolution is centered on the acquisition of real-time information. Real-time data is rare in the real estate industry. The Real Estate Transaction Standard (RETS) was introduced around 20 years ago to replace the industry’s current practice, which is frequent polling.
You ask a server to poll you if there has been any change in its data set since your last check. It replaced data downloading via manual file transfer protocol (FTP), a huge upgrade at the time. After two decades of operation, the challenges of polling have increased.
Unlike the old days when servers were polled four times per hour, systems now poll every 15 seconds. A data consumer polling 500+ MLSs every fifteen seconds is almost 2.9 million actions per daily. This data slamming has become more frequent in a competitive business environment. It has a number of business costs, product, operations, speed, and product costs. This has led to data quotas being imposed on data providers and rate limiting, which can cause frustration for all parties.
The RESO Transport Workgroup created RCP-027 (RESO EntityEvent Resource, Replication Model and Replication Resource) and RCP-018 (Push Replication Using Webhooks, and the RESO EntityEvent Resource) as a result of direct feedback received from data providers, vendors, and consumers.
Apps can use webhooks to send messages to other apps whenever an event occurs. If there is any change, the consumer will be pinged. This is also known as “push notifications” and most people who have modern smartphones are familiar with it.
Constellation1 was overjoyed to begin building an API to account RCP-027/RCP-028. These additions were completed in hours and not days. Kevin Yao, Senior Cloud Engineer at Constellation1, explains how they did it.
Their architecture is built on Amazon Web Services (AWS), which allowed them to add new features without having to modify their existing Web API system. Data is gathered from MLSs and processed through a data pipeline before being consolidated into a single stream (including member data, property updates, open houses, etc.). ).
They then created a simple notification service that taps into webhook events. Clients can then set up their APIs and receive the information.
Push notifications are sent to a URL via the subscription. Yao suggested this model in order to avoid an overwhelming number of communications such as email notifications.
The payload contains identifying information that allows the consumer to pick up the new data. This includes the record’s unique ID and source. It also contains a URL with its location.
Constellation1 is already learning by its own creation. They have provided a test endpoint so clients can try the system before they commit to it. They also asked for feedback to assist with improvement, advancement, and increased adoption. These are some of the items they are currently considering:
Full payload delivery
Additional security (e.g. message-level encryption).
Subscription pause
Retry settings can be customized
Redfin, RentSpree, and RE/MAX were some of the first to test the new system.
Data vendors and data consumers want the same thing: accurate and current information at the lowest possible resource cost. Inefficient and expensive polling that produces the same results for about 80 percent of the time is called mindless polling. Polling comes with hefty prices for Lambda functions and step functions, logs, storage, memories, reporting, and rights.
It is time to get beyond the Blackberry-centric methodology. Rick at Constellation1 (rick.herrera@constellation1.com) and RESO (dev@reso.org) are ready to be your industry partner toward a more efficient future of real estate data.