Is your club website being scraped?

Most Australian clubs discover scraping the same way: the secretary's address, which exists in exactly one place on the internet, starts receiving very specific spam.

Sales pitches for uniform suppliers. Offers to build the website you already have. Sometimes worse — a message that names your club, your president and your bank, and asks the treasurer to change some payment details.

None of that requires a hack. It requires a program that reads your public pages and writes down what it finds, which is a thing that happens to every website with an address, all the time, and which you are not going to stop.

This guide is about the version of the problem you can actually do something about. It starts with the honest limit, because every piece of advice below follows from it.

What scraping actually is

Scraping is automated copying of pages that are already public. A program requests your pages the same way a visitor's browser does, reads the text and images that come back, and stores whatever it was told to look for — email addresses, names, phone numbers, prices, dates, photos.

It is worth being clear that this is not a break-in. Nothing is bypassed. A scraper sees exactly what an anonymous visitor sees, and nothing that a visitor cannot see. That is both reassuring and the whole problem: the boundary of what can be scraped is the boundary of what you have published.

Plenty of scraping is entirely benign and you rely on it. Search engines are scrapers. So is the service that generates a link preview when someone shares your fixture page in a group chat. The distinction that matters is not automated versus human, it is what the collector does with what it takes.

Why club websites get harvested in particular

Community club sites are unusually attractive to harvesters, for reasons that have nothing to do with your club being important.

• Committee email addresses in plain text. A page listing the president, secretary and treasurer with their addresses is a ready-made contact list for cold sales, and a ready-made target list for the kind of fraud that impersonates a committee member to the treasurer.
• Real names attached to real roles. Enough to make a convincing approach. "Hi, it's the president, I'm at the ground and can't get to a computer" is a much better scam when the sender knows both names.
• Junior team lists. Names, age groups, training times and a venue. This is the category that should make a committee genuinely uncomfortable, and it is often published without anyone deciding to.
• Sponsor and supplier contacts, harvested by people selling to the same market.
• Venue, fixture and event data, collected by aggregator sites that republish local sport listings, sometimes with your photos attached.
• Photographs, taken for content farms and stock reuse, occasionally with the caption naming the child in the picture.
• Text for training and reuse, by large-scale crawlers gathering published writing at volume.

Notice how much of that list is people-data rather than club-data. Nobody is scraping your canteen price list to hurt you.

The honest limit

Here is the part most advice avoids saying plainly.

You cannot prevent bulk copying of a public page without also preventing search engines from reading it. They are the same request. A tool that reliably blocks the harvester also blocks the crawler that puts you on the results page when a parent searches for junior registration. Whatever protection you add, a determined collector can visit your site with an ordinary browser, look like a person, and take everything.

That means "stop scraping" is not an achievable goal and any product promising it is overselling. The two goals that are achievable:

1. Reduce the value of what can be taken — by publishing less personal information in the first place.
2. Notice when harvesting has happened, so you can act on the consequences rather than being surprised by them.

Everything below serves one of those two.

Publish less about people

This is the entire game, and it costs nothing but a decision.

Names, especially children's

Ask of every published name: does a stranger need this? A team photo caption naming every under-11 player, a results table with full names, a "welcome our new junior members" post — each one is a decision, and mostly it has never been made deliberately.

Reasonable defaults many clubs land on: first name and initial for juniors, no surnames alongside training times and venue, no birthdates at all, and photographs published under an image and consent policy that families have actually seen. None of this stops a parent finding their child's team. All of it removes the thing a harvester wants.

Contact addresses

Publish role addresses, not personal ones. secretary@yourclub.org can be reassigned at handover, filtered when it attracts spam, and abandoned without affecting anybody's personal mail. A volunteer's private address, once harvested, follows them for years after they leave the committee. Our guide to club email addresses covers setting these up.

Better still, put a contact form in front of the address so nothing is published at all. A form is not scraper-proof — nothing is — but there is no address on the page to collect.

Anything that does not need to be public

Member directories, contact lists, meeting minutes with personal details, medical or emergency information, and payment status all belong behind a login. Content behind authentication is genuinely out of reach of ordinary scraping, and this is the one technical control that actually works. It works precisely because it stops being publishing.

What the technical measures really do

Be clear-eyed about each of these, because they are routinely oversold.

• robots.txt. A polite request, not a control. Well-behaved crawlers honour it — including several of the large-scale collectors, which publish the names their crawlers use so you can exclude them. Harvesters ignore it entirely. Useful for managing the crawlers that cooperate; worthless as protection against the ones that do not. It is also public, so never list "secret" paths in it.
• Obfuscating email addresses. Splitting an address up or rendering it as an image defeats the crudest collectors and nothing else. It also makes the address harder for the people you want to hear from, including anyone using a screen reader. Use a role address or a form instead.
• Blocking right-click or disabling text selection. Purely theatrical. It irritates members and stops no software whatsoever.
• Rate limiting and bot filtering. Genuinely raises the cost of a fast bulk crawl and is worth having where your platform provides it. A patient collector spreading requests over days walks straight past it.
• Terms of use forbidding scraping. No technical effect at all, but worth publishing: it establishes that copying is not authorised, which matters if you ever need to make a complaint.
• Watermarking photographs. Does not prevent copying. Does make a copy traceable, which is sometimes all you need.

How to tell whether it is happening

You will rarely catch a scraper in the act. You catch the consequences.

• Plant a canary address. Create a unique email alias used in exactly one place — one page of your public site — and nowhere else. If mail starts arriving at it, you know exactly where the sender got it. This is the single most informative thing on this list and it takes five minutes.
• Search for your own sentences. Take a distinctive line from an unusual page and search for it in quotes. Republished copies of club content turn up surprisingly often.
• Reverse image search a club photo, particularly one that has been widely shared.
• Read your traffic reports for shape, not volume. Bulk crawls look wrong: hundreds of pages in a few minutes, no images loaded, an odd hour, one source, a visit pattern no person would produce.
• Watch what arrives at role addresses. The address that only ever appears on your committee page suddenly receiving industry sales mail tells you the page has been harvested.
• Ask members. People notice their own name in odd places long before the committee does.

When you find your content somewhere else

Separate the two cases, because they call for different actions.

Your content, republished. Annoying, occasionally damaging to your search position. Ask the site to remove it, then complain to whoever hosts it if that fails; search engines also accept reports about copied material. Keep a dated record of what you found.

Your members' personal information, republished. This is not a content dispute, it is a privacy matter, and it should be treated seriously rather than shrugged at. Write a removal request naming the specific information, keep the record, and be aware that privacy law where your club operates may give the people involved rights you can help them exercise. If children's information is involved, tell the families before they find it themselves, and take advice from your governing body.

Fraud attempts. If a harvested committee list is being used to impersonate your officers, warn the committee immediately, put a rule in place that payment detail changes are always confirmed by phone, and report it to the relevant authority in your area.

Tell your members what will be public

This is the obligation that outlives the technical discussion. People give a club their details to be members, not to be published. Say plainly, on the join form and in your privacy policy, what appears publicly — names on team lists, photographs, results — and give a straightforward way to ask for something to be withheld. A club that can answer "why is my child's name on the internet?" with a policy the parent already agreed to is in a completely different position from one that cannot.

Doing it on ClubHelix

The useful controls are the boring ones. Member records live behind authentication rather than on a page, roles and permissions decide which volunteers can see personal details at all, the member directory can be members-only or public as a deliberate choice rather than a default, contact goes through a form instead of a published address, and the audit log records who changed what. Member conversations run in member chat rather than on public pages.

What we will not claim: no platform can stop your public pages being copied, ours included. What a platform can do is make the private things genuinely private by default, so that the copying is of your fixtures and your news rather than of your people.

If you want a hand reviewing what your current site publishes about members, our done-for-you service includes that review as part of a migration. Otherwise start free and set the boundary deliberately from day one.

Frequently asked questions

Can we block scrapers from our club website?

Not reliably, and not without also blocking the search engines that bring you members. Anything a visitor can see, a determined collector can take. Treat prevention as impossible and put your effort into publishing less personal information and detecting harvesting when it happens.

Does robots.txt protect our members' details?

No. It is a request that cooperative crawlers honour and harvesters ignore. It is useful for managing well-behaved crawlers, including several large-scale collectors that publish their crawler names. It provides no protection at all for anything you would be upset to lose.

Should we publish our committee members' email addresses?

Publish role addresses such as secretary@yourclub.org, or a contact form, rather than anyone's personal address. Role addresses can be reassigned at handover and abandoned if they attract spam; a volunteer's personal address, once collected, follows them long after they leave.

Is it safe to publish junior team lists?

Publish as little as the purpose requires. First name and initial is enough for a parent to find their child, and much less useful to anyone else. Avoid publishing full names alongside ages, training times and a venue in the same place, and make sure families know in advance what appears.

How would we know if our site had been scraped?

Plant a unique email alias on one public page and nowhere else, then watch what arrives at it. Alongside that, search for a distinctive sentence from your site in quotes, reverse image search a well-known club photo, and look at your traffic reports for bulk-crawl shapes rather than raw visitor numbers.

Someone has republished our club's content. What can we do?

Ask them to remove it, then raise it with their host if they do not, and report copied material to search engines. If the copied material includes members' personal information, treat it as a privacy matter rather than a content one: make a removal request, keep dated records, and tell the people affected.