What's new at ElfHosted?

Symlinks get Supercharged

When we started using symlinks for real-debrid content, the natural place to store them was in /storage/elfstorage, since all users get 100GB free, so it's a consistent location. As we've increased in users, and users have increased their libraries sizes, I realized that this may be an inefficient way of "storing" the symlinks.

ElfStorage is backed by HDDs, and is using Ceph's erasure coding at a profile of 8:2, meaning every object gets split into 8 checks, and distributed across our 10 ElfStorage "dwarf" nodes (2 coding chunks). This is great for 80GB movies (10GB per node!), but probably not great for tiny little symlinks, since it represents a lot of extra work for the ceph cluster!

To this end, I've added a /storage/symlinks folder for each user. Unlike /storage/elfstorage, this is backed by NVMes on our 10Gbit nodes, in a simple 2-replica configuration, so the actual load on the cluster resulting from storage to /storage/symlinks should be negligible.

Your old symlinks in /storage/elfstorage will continue to work, but ElfBot will new create all new symlinks into /storage/symlinks. Apps default configs will be updated for fresh installs, but you'll want to update your Radarr / Sonarr root folders if you use them as part of your symlink setup.

ElfBot reports on broken symlinks

Many users with large Real-Debrid collections have been noticing that their content occasionally "goes away". The symlinks remain, but the original sources in /storage/realdebrid-zurg/__all__ (for example) are removed. We're not sure exactly why this happens to some content and not others, but to make it easier to manage, ElfBot can now generate you a report of all broken symlinks.

To get your report, run elfbot symlink report-broken, and find the resulting report in /storage/symlinks/report-broken.txt.

Today's scoreboard

Metric	Numberz	Delta
Total subscribers	316	+43
Storageboxes mounted	18	+1
Zurg mounts:	129	+21
ElfStorage in TBs	62	+5
Tenant pods	4059	+402
Elf-vengers	4	-
Trainees	2	-
Bugz squished	1	-
New toyz	1	-

Summary

Thanks for geeking out with us, and please share these posts with related geeks!

Full service outage on 29 February 2024 between 6:00 AM and 8:00 AM (CET)

We've received notice from Hetzner that they'll be undertaking maintenance on the switch which connects our 10Gbps "goblin" nodes, providing all of the shared storage for our /config volumes. The maintenance is scheduled for 29 February 2024 between 6:30 AM and 7:30 AM (CET)

Given the criticality of these nodes, and the amount of #chaos which would ensue if they all went away simultaneously, we'll be taking all customer workloads down 30 minutes prior to the scheduled maintenance, and bringing them back again 30 minutes afterwards.

This means that for ElfHosted customers, the actual service outage will be a total of 2 hours, from 6:00 AM to 8:00 AM (CET).

Time to 2-hour full service shutdown

In summary..

Xtremio Excels Excellently

After our recent hosting of public TorrentIO / Knightcrawler and Annatar instances, I was approached to take over hosting of another community Addon, the "Xtremio IPTV" addon. This one doesn't use Real-Debrid at all, but if you plug in your paid IPTV subscription details (assuming your provider supports xtreme codes), you can stream all your IPTV content inside Stremio!

It was a fun little addon to integrate, since there's no persistence or database required - you can see the final result at https://xtremio.elfhosted.com, and the public traefik stats here.

We do now offer hosted instances of Xtremio, which provide the same features, but with a higher rate-limit.

Annatar Arrives "Afficially"

A casual offer of hosting to a new addon announced in r/StremioAddons has turned into rewarding partnership with Annatar, a Stremio addon which leverages Jackett and Redis to provide a cached, just-in-time torrent search for Stremio.

The official, community site is at https://annatar.elfhosted.com, and we (@brett and I) have been geeking out over the public "GrafAnnatar" metrics.

Once the public instance was up, I added ElfHosted bundles to the store - these private instances benefit from the shared redis cache, which is well-seeded by the public instance, but allow users to configure their own selection of torrent indexers in their own Jackett backend.

AllDebrid 100% supported (if you BYO VPN)

We've had AllDebrid support for our debrid-related apps in testing for a few days, and our elf-a-testers have confirmed it's good to go (thanks elf-a-testers!).

The trick with AllDebrid, it turns out, is that they don't allow use of their API or WebDAV endpoints from "datacenter IPs", which includes our Hetzner IP ranges. (Confusingly, they allow listing of WebDAV files from anywhere, but not actual downloading!).

However, if you configured your AllDebrid account to specify that you're using a VPN, then you can access their endpoints via that VPN provider. Soo.. the workaround to using AllDebrid with ElfHosted apps is to... BYO VPN credentials (like we do with our torrent clients).

Currently this is just working with PrivateInternetAccess (PIA), but we can easily add more VPN flavors in future (let me know if you need one ASAP)

Here are the AllDebrid / VPN-enhanced versions of our regular debrid apps...

Elfbot Symlinks Save Storage Clutter

If you've been using your Real-Debrid account outside of the RDTClient+Arrs symlink solution (say, with Debrid Media Manager, or Stremio), you'll have lots of content sitting in /storage/realdebrid-zurg/{movies,shows} which is not symlinked to /storage/elfstorage, and thus not sorted into libraries managed by Radarr / Sonarr.

You can work around this by adding additional folders to your Plex libraries, but it's counter-intuitive, and messy, and it can be especially hard to track watched vs unwatched TV episodes, etc.

For media which did arrive via the Aars / RDTClient, the symlink design feature of RDTClient is genius: you end up with a 2kb file instead of a 60GB file, which you can rename and move around your filesystem, sort into folders using the Aars, and delete and replace when an upgrade becomes available, etc.

We can now replicate the symlink design using ElfBot, to import all your existing Debrid content (not sourced via RDTClient) into Radarr / Sonarr for better organizing / managing!

For details, check out these pages:

Scannarr added to Radarr / Sonarr

You might have noticed your Radarr / Sonarr complaining about the lack of RSS-capable indexers. This is because, unlike some popular trackers, the Prowlarr knightcrawler / stremio indexer doesn't (and can't) include an RSS feed.

Sonarr especially relies on an RSS feed to alert it to when new episodes are out, and ready to search. Without an RSS source, you have to manually trigger searches for your TV series, in order to grab new episodes when they're downloaded.

Thanks to Scannarr, we now have a solution!

Elf-Disclosure for Jan 2024 now available

We're 👶 6-months old now, and I've just finished our first Elf-Disclosure report from 2024!

Here's an excerpt:

January saw us pivot away from "traditional" seedbox hosting and towards the exciting debrid-based "infinite streaming" space. You'll notice that while hardware costs remained mostly stable (this will not be true in February, I'm afraid!), development costs (our most costly expense!) doubled, as we introduced a slew of new apps and guides to capitalize on this increased interest.

You can read about all the changes, expenses vs income, and geek out over the tech stats, in the Elf-Disclosure for Jan 2024 report!

AllDebrid Alternatives Available

It's been a week since we announced our free public torrentio alternative (with paid private instances), and it's been exciting (and at times, overwhelming!) to welcome a flock of enthusiastic new elves to the platform.

The influx of users has brought some edge-cases / bugs to the fore, which I'll detail below..

Stremio Scaling Stresses

In our previous episode, your brave (perhaps foolhardy?) elves had scaled the infrastructure and the humans, preparing to announce our tiered torrentio-like service (free-public, paid-subscription, and free-internal), https://torrentio.elfhosted.com, to the rampaging streaming hordes at r/StremioAddons.

The next morning iPromKnight PR'd a huge (and hugely more efficient) replacement of the original scrapers, so I scrambled to YOLO-implement these scrapers into our instance before launching.

The announcement post went flawlessly, except in my rush I misspelled "ElfHosted" in the title!

In the 3 ½ days since the announcement, we've added 39 more (free ElfBuckz) subscribers, and 35 of those are torrentio subscribers.

The extra attention have added stress in a few areas, which I'll detail below..