Protecting an online shop from scraping and bots
Contents
Share
Protecting a shop from bots starts with the names of the events, not with one “antibot” switch. Scraping takes content in order to use it somewhere else. Scalping takes a scarce good in a way a person cannot repeat by hand. Inventory denial fills a cart without payment. Stolen login-password pairs are tested at sign-in. Routine invalid traffic, in the MRC standard, is recognized with lists. Click fraud is not in that set: a click on an ad does not copy a product card.
Below is a breakdown by shop surface and an order that does not mix the events. These sources have no figure for “what percent of orders are bots,” and none is added here.
Four events that get collapsed into one report
OAT-011 Scraping is collecting application content and other data for use somewhere else. The description is about copying available data and processed output. Example purposes include exploiting the application’s own data and affecting prices. Collection can run continuously or in bursts, with a fake or taken-over account, or with no sign-in. If another application stands between the person and this one, the catalog points to a different event. If the goal is to obtain money or goods, it points to another event as well. This entry does not measure a drop in the original site’s rankings. The path order is not part of this description.
For a shop, that is the card, the price, and the stock level ending up on someone else’s storefront. A customer sign-in is not required.
OAT-005 Scalping is obtaining goods or services of limited availability in a way an ordinary user could not do by hand. Watching for an item to appear and acting quickly can be part of the event. The catalog separates it from last-second sniping and from speeding up a routine on the user’s behalf. An extra sign is that the good is limited. The catalog’s example is tickets later resold for more. The consequence: the good quickly becomes unavailable to other people. Named examples include booking tables and rooms, and a bot on slots.
OAT-021 is selecting and holding items that are never bought, paid, or confirmed, so other people cannot buy them. The attacker does not receive the good. The ordinary example is many items in a cart with no payment. The same technique covers unpaid reservations, slots, queues, and quotas. If what runs out is server resources, that is denial of service, not scalping and not scraping.
Sign-in is separate. Cracking is many passwords against one account. Credential stuffing is testing pairs leaked from another site. Spraying is one weak password across many accounts. OWASP calls a second factor the best defense against most of these attacks. A compromise share with no year and no sample is not used in the calculation.
| Shop surface | Event | What you lose | What the event does not do |
|---|---|---|---|
| Catalog and price | Content collection | Data leaves, prices may be affected | It does not measure a ranking drop and it is not a purchase |
| Scarce goods | Scalping | A unit goes to a method that cannot be repeated by hand | It is not an unpaid cart |
| Cart and booking | Inventory denial | Others cannot buy; the attacker does not receive the good | It is not a server failure when memory runs out |
| Account sign-in | Stolen pairs, cracking, spraying | The account | It does not copy the storefront |
Routine traffic that a list can still recognize
The MRC addendum (final version, June 2020) splits invalid traffic. The class recognized routinely is recognized by lists or by standard parameter checks. The examples include known data-center traffic with consistently invalid activity, but not the route artifacts of live users; bots, spiders, and other crawlers; checks on transaction parameters; a non-browser User-Agent; prefetch after which a person never opened the ad; sizes 0×0 and 1×1; sessions that cannot render an image, including a headless browser.
The list is for an MRC accreditation audit. It is not mixed with another taxonomy and not equated with the share of such traffic in Yandex Direct. For a shop the conclusion is narrow: a crawler and a data center in this list are routine recognition, and scalping and stolen passwords are not on the list. A data-center list cannot close price copying: the scraping entry explicitly allows collection with no sign-in and with a taken-over account, not only from a known data center.
Order for the shop
Name the complaint. “The price was copied” is content collection. “The limited run was taken in seconds in a way that is not by hand” is scalping. “Carts sit unpaid” is inventory denial. “They sign in with other people’s passwords” is stuffing, not scraping.
For collection, do not promise that a data-center list will close it. Collection runs with no sign-in and with someone else’s account, continuously or in bursts.
On a scarce item, do not call an unpaid cart a purchase. The controls differ because in one case the attacker receives the good and in the other they do not.
If server memory fell over, that is not scalping: the catalog points to denial of service.
At sign-in, separate cracking, testing leaked pairs, and spraying. A second factor is the defense against most of these attacks, without someone else’s percentage.
Check the ad click separately: who clicks ads, click fraud and sklikivanie, invalid clicks, a Yandex refund. The product card does not make those decisions.
Launch checklist:
The catalog has an answer to price copying, separate from account sign-in.
The limited run has an answer to obtaining the good, separate from an unpaid cart.
An unpaid booking returns the slot to other people, and it is not labeled “we were bought out.”
Sign-in is not protected only by a data-center IP list from an advertising standard.
A second factor sits on most sign-in attacks, not on a “leave a review” form.
The ad report is not filed as proof of scraping.
The manual limit: by eye you can tell one foreign storefront and one cart holding the whole stock. A stream of copied cards and a stream of pairs from a leak cannot be read by hand. Binetro looks at the visit on the advertiser’s site and can allow, challenge, or block. It does not promise to close price scraping entirely, and it does not refund a click: the ad platform charges the click.
Collect the shop’s week as four complaints and do not glue them into “bots.” A complaint about someone else’s storefront with your prices is content collection: available data and processed output are copied, continuously or in bursts, with no sign-in or with a taken-over account, and affecting prices is among the purposes. The catalog does not measure a ranking drop. A complaint that “the limited run was taken in a way that is not by hand” is scalping. The catalog’s example is tickets later resold for more, and the good quickly becomes unavailable to other people. A complaint that “carts hold the whole stock with no payment” is inventory denial: the attacker does not receive the unit, and the same applies to reservations, slots, queues, and quotas. A complaint that “they sign in with passwords from another site” is stuffing, beside cracking many passwords against one account and spraying one weak password. OWASP calls a second factor the best defense against most of these sign-in attacks. It does not transfer to the product card: the card is copied with no sign-in.
The MRC list does not close that week with one address file. It is for an advertising audit: known data centers with consistently invalid activity, crawlers, a non-browser User-Agent, prefetch with no ad opened, 0×0 and 1×1 pixels, sessions that cannot render an image, including headless browsers. Scalping and stolen passwords are not on that list. The list does not set a share in Direct. A crawler from the list and a buyer with a leaked pair are different rows of a report.
If server memory ran out during the day, do not write “we were bought out” in the review. The inventory-denial entry points to denial of service in that case. If the good did reach people by a method that cannot be repeated by hand, that is scalping, even when an unpaid cart sits beside it: two events in one release.
Keep four labels on the wall and do not add a fifth called “just bots.” The catalog label answers copying. The scarce-goods label answers obtaining the good. The cart label answers holding without payment and returning stock to other people. The sign-in label answers cracking, leaked pairs, and spraying, and it places a second factor against most of those attacks, with no percentage from someone else’s analysis. The MRC advertising list hangs beside them as a reminder that a crawler and a data center are recognized routinely for an advertising audit and replace none of the four labels. If a label is empty, the week’s complaint stays without a control, even when an “antibot” switch is on.
One product release can touch two labels at once. Tickets went to people who acted in a way that cannot be repeated by hand — scalping. Unpaid reservations sit beside that — inventory denial. Server memory fell over — denial of service. Three lines in the log are more honest than one sentence, “bots attacked us.” An effect on prices belongs to content collection and does not prove the site’s rankings fell: the catalog does not measure that.
Keep the advertising click as a fifth line that does not heal the first four. Who clicks an ad, how click fraud differs from sklikivanie, when a click is judged invalid, and whether the money comes back are platform decisions. Copying a card, scalping a ticket, a cart left hanging, and someone else’s password do not decide those. The routine MRC list is useful when the log shows a known data center, a crawler, a non-browser agent, prefetch, a 1×1 pixel, or a session with no image. It is useless as an answer to “why was the cart not paid.”
If only one label is empty for the week, start there, not with a general ban list. An empty sign-in is a second factor against most password attacks. An empty scarce-goods label is the difference between “the good was obtained” and “the good was only held.” An empty catalog is data collection, which can happen with no sign-in. An empty cart is stock returned after non-payment. That is how a shop review stays complete even when none of these sources gave you a bot share of orders.