Skip to content

RadioOS

RadioOSBot

RadioOSBot is the crawler behind RadioOS, software that radio stations use to run their day. It reads the public news feeds of local outlets and public agencies, so a host can see what a community is talking about before going on air.

If you run a site and want us gone, you can skip to How to stop us. It takes one line, and we will not ask why.

How to recognise it

Every request carries this name:

RadioOSBot/1.0 (+https://radioos.ca/bot)

It also sends From: sources@radioos.ca. It comes only from the addresses below. Each one's reverse DNS is bot.trends.radioos.ca, and that name resolves back to the same addresses, so you can check a request the way you would check a search engine's crawler. Anything else using our name is not us.

167.114.66.156
2607:5300:205:200::1:117

What it reads

  • robots.txt, and the two files that say what a site allows machines to do, /.well-known/tdmrep.json and /ai.txt, once a day.
  • One public RSS or Atom feed per outlet, and news sitemaps where a site has no feed.
  • Your home page and your terms of use, about four times a year, to see whether the terms say anything about reading your feed. If they do, we stop, and a person reads them.
  • Environment Canada's weather alerts, each one as its Datamart announces it, through the Datamart's own subscription service.
  • Bluesky's public stream of new posts, and its trending topics once an hour, to count how often public figures and provinces are named.
  • When we first come across a site, and again after 30 and 90 days and then twice a year if it had none, to find out whether it has a feed: its robots.txt, its home page once, and the usual addresses for a feed or a news sitemap (/feed/, /rss, /rss.xml, /feed, /index.xml, /atom.xml, /news-sitemap.xml, /sitemap-news.xml). It stops at the first feed it finds. Across the whole web this finding-out is at most 1 request a second.

It never reads article pages. It never crosses a login, a paywall or a challenge page, and it does not run a browser.

What it keeps

From each item in a feed it reads four things: the title, the link, the date and the description. From each item in a news sitemap it reads three: the title, the link and the date. From each Environment Canada alert it reads three: the headline, the time it was sent and the provinces its areas lie in. It skips the rest, including full text and images, without reading them. From Bluesky it keeps only a count each minute and the trending topics that are a public figure's name or plain words: never a post, its text or who wrote it.

  • On disk: the link, the outlet, the times, and facts worked out from the item, such as which places it mentions. No title, no description and no article text. An Environment Canada alert's headline is the one exception, because its licence allows it to be kept and shown unaltered.
  • In memory only: the title, for up to 24 hours, and the description, for as long as it takes to process the item.

What a station sees is our own words about a story, with your outlet's name, the time and a link to you. We show a publisher's own headline or text only when a licence or the publisher's written yes allows it.

How it behaves

  • At most one request to a host every 10 seconds, and fewer if your site asks.
  • At most 2 requests a second to one server address, so sites sharing a server are not hit together. Across everything it reads, at most 50 requests a second.
  • One connection to a publisher at a time.
  • robots.txt is read at most once every 24 hours. Rules for RadioOSBot and for * both apply. If robots.txt answers with a server error, we read nothing from that host until it recovers.
  • Crawl-delay and Retry-After are honoured. A 429 or a 503 makes it back off, doubling each time, up to 24 hours between tries.
  • If one server address turns it away (a 401, 403 or 451) on 5 of the sites it hosts within 15 minutes, it waits on every site at that address, from a minute, doubling each time, up to an hour between tries.
  • It asks for gzip, sends conditional requests, and adds nothing to your addresses to get around a cache. A host that stops resolving is tried weekly, then monthly.

How to stop us

Either of these works. Neither needs a reason or a conversation.

Add this to your robots.txt. It takes effect within 24 hours.

User-agent: RadioOSBot
Disallow: /

Or email sources@radioos.ca with your domain. Reading stops the same business day, and it stays stopped until you tell us otherwise.

We also stop if your terms bar automated reading of your feed, or if you reserve text and data mining in a machine-readable way where the law gives that force.

Questions

sources@radioos.ca reaches a person. Tell us if the bot has done something this page says it does not.

Back to RadioOS

Request a quote

Tell us about your stations and what you’d like to run. We’ll reply personally with a quote.

  1. 1
  2. 2
  3. 3

We use these details only to reply to you. No mailing lists. Privacy

Sign in to RadioOS

Station staff sign in at their company’s own RadioOS address.

.radioos.ca

Not sure? Any email from RadioOS has links that take you straight to your Backstage, or ask your station’s administrator.

From the studio to the dial

On air

A station agent that keeps working through outages, off-air recording, streaming and emergency alerts. Call-in lines and remote audio are coming soon.

See how it works

What’s included

  • A self-updating station agent, with failover pairs for redundancy
  • Audio logger: continuous off-air recording and your CRTC archive
  • HLS streaming with sound processing and stream ad replacement
  • Alert Ready emergency alerts, with airing proof and compliance reports
  • Transmitter sites, equipment and maintenance in one place

Add to your quote

  • Studio phonesSIP lines, call screening, a caller queue and the studio consolePer stationComing soon
  • Outpost remote codecLive audio from a remote or venue over the internetPer stationComing soon
  • Backup audioA silence watch with alarms, and a backup list that fills the gapPer stationComing soon
  • FM/AM processorProcessing and the stereo generator for each station. Needs STLPer stationComing soon
  • RDS encoderThe RDS signal, generated from what’s playing. Needs Now playingPer stationComing soon
  • STLYour programme to the transmitter site over IP, FM or AM, with a bypass at the sitePer stationComing soon
  • AI HostOvernight breaks written and voiced for each station, sent to your automationPer stationComing soon