Here I am, running separate tailscale instances and a separate reverse proxy for like 15 different services, and that’s just one VM… All in all, probably 20-25 tailscale instances in a single physical machine.

Don’t think about Tailscale like a normal VPN. Just put it everywhere. Put it directly on your endpoints, don’t route. Then lock down all your services to the tailnet and shut down any open ports to the internet.

My NAS will stay on bare metal forever. Any complications there is something I really don’t want. Passthrough of drives/PCIe-devices works fine for most things, but I won’t use it for ZFS.

As for services, I really hate using Docker images with a burning passion. I’m not trusting anyone else to make sure the container images are secure - I want the security updates directly from my distribution’s repositories, and I want them fully automated, and I want that inside any containers. Having Nixos build and launch containers with systemd-nspawn solves some of it. The actual docker daemon isn’t getting anywhere near my systems, but I do have one or two OCI images running. Will probably migrate to small VMs per-service once I get new hardware up and running.

Additionally, I never found a source of container images I feel like I can trust long term. When I grab a package from Debian or RHEL, I know that package will keep working without any major changes to functionality or config until I upgrade to the next major. A container? How long will it get updates? How frequently? Will the config format or environment variables or mount points change? Will a threat actor assume control of the image? (Oh look, all the distros actually enforce GPG signatures in their repos!)

So, what keeps me on bare metal? Keeping my ZFS pools safe. And then just keeping away from the OCI ecosystem in general, the grass is far greener inside the normal package repositories.

the H200 has a very impressive bandwith of 4.89 TB/s, but for the same price you can get 37 TB/s spread across 58 RX 9070s, but if this actually works in practice i don’t know.

Your math checks out, but only for some workloads. Other workloads scale out like shit, and then you want all your bandwidth concentrated. At some point you’ll also want to consider power draw:

• One H200 is like 1500W when including support infrastructure like networking, motherboard, CPUs, storage, etc.
• 58 consumer cards will be like 8 servers loaded with GPUs, at like 5kW each, so say 40kW in total.

Now include power and cooling over a few years and do the same calculations.

As for apples and oranges, this is why you can’t look at the marketing numbers, you need to benchmark your workload yourself.

Well, a few issues:

• For hosting or training large models you want high bandwidth between GPUs. PCIe is too slow, NVLink has literally a magnitude more bandwidth. See what Nvidia is doing with NVLink and AMD is doing with InfinityFabric. Only available if you pay the premium, and if you need the bandwidth, you are most likely happy to pay.
• Same thing as above, but with memory bandwidth. The HBM-chips in a H200 will run in circles around the GDDR-garbage they hand out to the poor people with filthy consumer cards. By the way, your inference and training is most likely bottlenecked by memory bandwidth, not available compute.
• Commercially supported cooling of gaming GPUs in rack servers? Lol. Good luck getting any reputable hardware vendor to sell you that, and definitely not at the power densities you want in a data center.
• TFLOP16 isn’t enough. Look at 4 and 8 bit tensor numbers, that’s where the expensive silicon is used.
• Nvidias licensing agreements basically prohibit gaming cards in servers. No one will sell it to you at any scale.

For fun, home use, research or small time hacking? Sure, buy all the gaming cards you can. If you actually need support and have a commercial use case? Pony up. Either way, benchmark your workload, don’t look at marketing numbers.

Is it a scam? Of course, but you can’t avoid it.

Ctrl+Alt+F2

Unless you have actual tooling (i.e. RedHat erratas + some service on top of that), just don’t even try.

Stop downloading random shit from dockerhub and github. Pick a distro that has whatever you need packaged, install from the repositories and turn on automatic updates. If you need stuff outside of repos, use first party packages and turn on auto updates. If there aren’t any decent packages, just don’t do it. There is a reason people pay RedHat a shitton of money, and that’s because they deal with much of this bullshit for you.

At home, I simply won’t install anything unless I can enable automatic updates. Nixos solves much of it. Two times a year I need to bump the distro version, bump the nextcloud release, and deal with depreciations, and that’s it.

I also highly recommend turning on automatic periodic reboots, so you actually get new kernels running…

Just going off the marketing here:

Git server with CI/CD, kanban, and packages.

From the looks of it, they also seem to bundle the vscode server and a bunch of other stuff. I’m actually kinda surprised they do it with only 1G of RAM.

Not to be that guy, but 12% of 8G isn’t even close to ”heavy as fuck” for a CI/CD and collaboration suite that seems aimed at enterprise users.

You can also tweak how much memory you’d like the jvm to grab with ’-Xms100m’. Any defaults are most likely aimed at much larger deployments than yours.

But yes, Java is a disease.

You mean ”hardcore WAF challenge”?

If you’ve taken care to properly isolate that service, sure. You know, on a dedicated VM in a DMZ, without access to the rest of your network. Personally, I’d avoid using containers as the only barrier, but your risk acceptance is yours to manage.

Well, I’d just go for a reverse proxy I guess. If you are lazy, just expose it as an ip without any dns. For working DNS, you can just add a public A-record for the local IP of the Pi. For certs, you can’t rely on the default http-method that letsencrypt use, you’ll need to do it via DNS or wildcards or something.

But the thing is, as your traffic is on a VPN, you can fuck up DNS and TLS and Auth all you want without getting pwnd.

Then you expose your service on your local network as well. You can even do fancy stuff to get DNS and certs working if you want to bother. If the SO lives elsewhere, you get to deploy a raspberry to project services into their local network.

I’d recommend setting up a VPN, like tailscale. The internet is an evil place where everyone hates you and a single tiny mistake will mess you up. Remove risk and enjoy the hobby more.

Some people will argue that serving stuff on open ports to the public internet is fine. They are not wrong, but don’t do it until you know, understand and accept the risks.(’normal_distribution_meme.pbm’)

Remember, risk is ’probability’ times ’shitshow’, and other people can, in general, only help you determine the probability.

Two things I never want to work with and will just pay someone else to deal with whenever possible:

• Email
• Printers

And that’s about it, almost everything else I’m fine doing myself.