Gathering and verifying accurate real-time data is essential to competing online. However, when an enterprise, retail company, travel site, or any other brand attempts to access a competitor’s site, they will likely be blocked or shown misleading content. Or Lenchner, CEO of Luminati Networks, explains how their residential proxy network solution enables users to access and gather the most accurate data available by seeing the internet through their consumers’ eyes.
This interview was originally published on October 11, 2020.
What is Luminati?
Luminati Network helps some of the world’s largest companies gather and collect publicly available data online in an ethical manner. To better understand this concept, we’ll use an example of two competing supermarkets. Supermarket A can’t compete effectively if they don’t know what Supermarket B is offering. So Supermarket A will send a “mystery shopper” to Supermarket B to see their prices, what products they’re promoting, and what’s on sale.
The same thing is happening online; only it is way more complicated. Because the web is so segmented, even if you and I visit the same page simultaneously, we will both see different things, as most sites try to present visitors with their best offers based on several factors. That’s fine for consumers, but a company cannot remain competitive when they see just one perspective of their competitor’s site when there are millions of other perspectives.
At Luminati, we allow our thousands of customers, including Fortune 500 companies, to view the web as if they are the consumers from every location globally on a vast scale. We serve around 25 different use cases, including eCommerce, finance, online travel, advertising, and brand protection, assuring big brands that counterfeit products are not being sold, and resellers are not breaching their policies.
How does Luminati allow viewing the web as consumers?
The problem begins when a website suspects traffic is from their competitor, or a fake consumer, and either blocks access or displays only segmented pieces of information. To see the entire web, you need to visit it on a mass scale from different IP addresses, and this is where proxy networks come into play. Previously, it worked fine to rent data center-based IPs from around the world. But the internet became more sophisticated, and utilizing third-party tools, can now identify IPs hosted by a data center.
So, in 2014 we initiated Luminati Networks and invented a residential proxy network solution. Every month, we have tens of millions of unique device owners who actively and knowingly allow us to route some traffic through their IP addresses while their device is idle, fully charged, and connected to an internet source, a majority of whom come through our partnerships with other app owners.
A Residential Proxy Network is a superior solution because it uses real IPs of real devices to visit websites, so there’s nothing suspicious about the activity, causing it to be blocked, allowing businesses to view the internet just like typical consumers do every day. And because we have so many residential IPs worldwide, we also solve the problem of getting different results based on parameters such as physical location. If you want to see the price shown to someone in Antarctica, go ahead and use one of our residential IPs in Antarctica. If you want to compare the prices offered to someone in New Jersey as opposed to someone in Chicago, that is also no problem.
And it’s not just segmentation; it’s also discrimination. We have seen users of Apple devices like Macs or iPhones get charged higher prices than Android or PC users. Showing our customers the truth allows them to compete by providing better prices to win these customers and maybe even end this discrimination.
Please take us through the Luminati process.
Anyone who wants to use the Luminati residential IPs network needs to undergo a very extensive vetting procedure, including KYC “Know Your Customer” procedures. Our group of compliance officers is required to approve every customer. Because we are structured by real user devices, we must ensure that every action is completely legitimate and secure before granting access to our network. This is a multi-phase comprehensive process.
Once an account is set up, customers set up their desired parameters – what IP and devices they want to use, the target domain, how the URL should be structured, what data to extract, and so on. Their requests are then sent to the other website through the Luminati proxy network infrastructure. That website responds again through the Luminati proxy infrastructure, so we are like a tunnel. Within the Luminati control panel, there are several dashboards to view, debug, and track the success rate of their traffic through our network.
How does Luminati help eCommerce sites maintain their competitive edge?
eCommerce is our biggest and most interesting use case. When consumers are looking to buy a smartwatch, they’ll usually visit two, three, or even four or more websites to compare prices. But their final decision is not based only on the lowest price but also on several other data points. Consumers will read product reviews and compare shipping speeds and costs. They might also trust a website with forty pages of smartwatch search results more than a site with only four pages of results.
While price will always be the number one data point to be collected, those other parameters are also important. It’s all about conversions, so our customers want to be the best in every factor that visitors consider when purchasing.
There is also SEO data that exists on eCommerce websites. You want to see what keywords people searched for on your competitors’ websites, so you will be able to also show the right results on yours.
It’s pretty much the same in the online travel industry when booking a flight, a hotel room, or car rental. All these websites want to win the customer, so they need to have the best rates, plenty of photos, and check customer reviews. So, they use Luminati to openly collect this publicly available data from around the web, including their competitor websites.
And not only does data collection need to be done on a mass scale, but also in real-time because if you repriced your product 24 hours ago, it’s probably already obsolete, and you’re no longer in the game.
How can companies structure and verify the authenticity of their online advertising using Luminati?
When it comes to marketing, you don’t only want to see how your competitors are pricing products; you also want to know the ads their consumers, who are also your potential consumers, are seeing. Advertising today is perhaps the most segmented and personalized aspect of the internet. To see what ads competitors are serving to different consumers, you need to come as those consumers. It’s not like driving on the highway and seeing the same billboard as everyone else. On the internet, everyone has a different highway.
In today’s AdTech ecosystem, there are way too many players. You have an attribution company doing the link tracking; you have affiliate partners, you have SDK (software development kit) partners, you have mobile app developers that embed the SDKs. Therefore, one ad’s link may go through ten different companies and their technologies or parameters, increasing the possibilities of consumers not getting the right ad, getting no ad at all, or even being directed to fraudulent content. A lot of advertising money can be wasted this way, so to catch these issues, you need to mimic real consumers. When you see something broken or fraudulent activity, you can approach the right company to get it fixed.
How do you help companies mitigate other types of fraud?
Platforms that sell third party products need to verify that products submitted for review are the actual product being sold to consumers. However, fraudsters use sophisticated methods to analyze IPs, and therefore will show fake content to traffic from IPs they suspect are coming from that company, while pushing their illegal content to consumers. The only way to catch this fraud is to present themselves as a potential consumer, i.e., possible victim, which can be done using our residential IPs.
How can Luminati help site owners be sure their website is functioning correctly all over the world?
Companies, especially global ones, need to continually check how their website is being loaded from different locations using different internet connections. They need to check the website’s localization. Is the translation working correctly? Does that user in China see the site as intended? What is the loading speed for a user in Atlanta with a 4G connection? Luminati can give you all that.
What are your Data Unblocker and Data Collector products?
In addition to working with our proxy infrastructure, customers can define a website they want to approach. Our very sophisticated Data Unblocker product will unblock it, allowing them to collect the data. This product has been out and working great for almost two years, and we’re happy to see that it’s helping make data more of a commodity, as one no longer needs to be a proxy expert to access the data they want.
Once we saw Data Unblocker working correctly and handling basically 80% of the work, we realized that we also need to take care of the full data extraction cycle. With our Data Collector, you don’t need to think on your own and work independently with our proxy network; or even work with our Data Unblocker. Just tell us what data you want to collect from which website – how many times a day for how long and in what format. Our automated Data Collector solution takes responsibility for 100% of the process. This product is currently in production and working very well for some of our large customers. Our vision is that in the next few years, no one will have to integrate with proxies directly. Customers will submit their data requests, and we will deliver the data.
Gail is a Technical Editor and Interviewer for Website Planet. Her first PC was a TRS-80 which required a cassette tape to boot up. Producing websites, emails, and banners combines her love for design, technology, and of course, writing.
Thank you, - your comment was submitted successfully!
We check all user comments within 48 hours to make sure they are from real people like you. We're glad you found this article useful - we would appreciate it if you let more people know about it.
Share this blog post with friends and co-workers right now:
Thank you, , your comment was submitted successfully!
We check all comments within 48 hours to make sure they're from real users like you. In the meantime, you can share your comment with others to let more people know what you think.
Thank you for signing up!
Once a month you will receive interesting, insightful tips, tricks, and advice to improve your website performance and reach your digital marketing goals!
