I’m not real clear what exactly you need to document.
Infrastructure documentation starts with an IPAM.
A good IPAM can help you document all kinds of stuff.

I use NetBox.
https://github.com/netbox-community/netbox?tab=readme-ov-file#getting-started

I’m running it as a Docker container on a Linux VM.
I just looked at their latest screenshots, and it appears they’ve done quite a bit with it since I stood up my copy.
It does even more now. I’ll have to upgrade.

Cool.

Here. SSH key issues. There was a huge forum war.
https://forum.proxmox.com/threads/ssh-keys-in-a-proxmox-cluster-resolving-replication-host-key-verification-failed-errors.138102/
But its still a thing. That still needs to be fixed by a human. Today that’s me.

Regarding CEPH and corosync on the same network … well I’m just getting started with that now. I do have them on different vlans, but its the same 10gb set of nics. I’m hoping if it gets really lousy, my netadmin can prioritize the corosync vlan. I’ll burn that bridge when I come to it.

EDIT … The linked forum post above leads to the SSH key answer, but its convoluted.
Here’s what I put in my own wiki.

Get the right key from each server.
cat ~/.ssh/id_rsa.pub

Make sure they match in here. Fix em if they don’t.
/etc/pve/priv/authorized_keys

There’s a couple symlinks to fix too, but this should get it.

SSH key management in PVE is handled in a set of secondary files, while the original debian files are replaced with symlinks. Well, that’s still debian. And in some circumstances the symlinks get b0rked or replaced with the original SSH files, the keys get out of sync, and one machine in the cluster can’t talk to another. The really irritating thing about this is that the tools meant to fix it (pvecm updatecerts) don’t work. I’ve got an elaborate set of procedures to gather the certs from the hosts and fix the files when it breaks, but it sux bad enough that I’ve got two clusters I’m putting off fixing.

Corosync is the cluster. It’s a shared file system that immediately replicates any changes to all members. That’s essentially anything under /etc/pve/. Corosync is very sensitive. I believe they ask for 10ms lag or less between hosts, so it can’t work over a WAN connection. Shit like VM restores or vmotion between hosts can flood it out. Looks fukin awful when it goes down. Your whole cluster goes kaput.

All corosync does is push around this set of config files, so a dedicated NIC is overkill, but in busy environments, you might wind up resorting to that. You can put cororsync on its own network, but you obviously need a network for that. And you can establish throttles on various types of host file transfer activities, but that’s a balancing act that I’ve only gotten right in our colos where we only have 1gb networks. I have my systems provisioned on a dedicated corosync vlan and also use a secondary IP on a different physical interface, but corosync is too dumb to fall back to the secondary if the primary is still “up”, regardless of whether its actually communicating, so I get calls on my day off about “the cluster is down!!!1” when people restore backups.

I use PVE professionally. I could spent some time bitching about how it handles ssh keys and the fragile corosync cluster management. I could complain about the sloppy release cycle and the way they move fast and break shit. Or all the janky shit they’ve slapped together in PBS. I could go on.

But I actually pay for a license for my homelab. And ya, it is THE thing at work now.

I’ve often heard it said that Proxmox isn’t a great option. But its the best one.
If you do try it, don’t bother asking questions here.
Go to the source. https://forum.proxmox.com/

Hmm. I used to volunteer with Free Geek in Portland OR. It was essentially that, an e-disposal site and we made refurbs for community organizations. But they did have a store for sale to the public.

I have so much computer junk. I got rid of most of it, but then I got a bunch more when we closed the company office. Got at least 10 monitors, 5 PCs, a mini, couple laptops … and a storage shelf to put it on.

Buy? That’s garbage. Look in garbage places. Used shops of any sort.
I like the college junk store suggestion. I used to do that.

You better watch it. You will shortly have a closet full of junk computer parts.

Do we still get to loot his stuff?

Why did I enjoy that so much? Right down to the mom’s look as she left. Sheathing the axe. Marginally solid.

Ok. Yes, my use case is a private document and media store. I’m ungoogling.
VPN seems like a good place to start. But I’d like a simple answer, and I expect there are none to be had. As you’ve illustrated here, I’ll find a reason to punch holes in the firewall. And then I’m going to need to secure a web server. Life happens. I’ll keep it simple for now while I sort things. Thanks for your perspective.

Ya. I understand VPN. I do enterprise IT stuff. The things I build assume a secure environment. VPN is step one.
Nailing down a web server on the internet tho … there’s so many ways to attack. There’s so many things to secure. And its a bit complex to manage all that.
The nextcloud site covers hardening the server, but doesn’t even mention vpn.
I’ve been watching threads like this. I’m pretty convinced vpn is the answer.

Well, I might as well put a dog in the fight. I’m considering my final, actually secure deployment of nextcloud.

This discussion has convinced me that a vpn is the only answer.
And almost everyone says wireguard.

K. Thats what I will build.

And this is the start of the longest crypto nerd fight I’ve seen on Lemmy. Well done, people!

Thanks. That’s well laid out, straightforward. I have resources at home that I want access to through my vps. This is a good blueprint.

RTFM. …
… The last thing you try.

Proxmox is Debian, so much of your ideas could translate directly across. That said, I try to mod the PVE server as liitle as possible.

Proxmox makes it so easy to spin up yet another VM or LCX to handle services with its core offerings. Also google “proxmox helper scripts” to find tteck’s additional stash of ready-made LCX.

Boy. You asked about Proxmox. Nobody said anything.

How does Proxmox make it easier? Have you used it? All sorts of ways. Like, its a full virtual infrastructure management system instead of just an OS. Proxmox loves ZFS. It does many of the things you’ve mentioned here.

Proxmox does have its own backup system that can work with an NFS target or with their smart dedupe storage and replication server product. https://www.proxmox.com/en/products/proxmox-backup-server/overview

You’ve got some pretty advanced ideas and perhaps have already moved beyond the Proxmox question. But if you are curious and haven’t used it, spin up a server and give it a whirl.

Do it. Jump in. Just start with whatever you can assemble.
It’s a great way to keep your room warm.

I don’t know about the Ubuntu LCX. I don’t container much.

I’d do this with a virtual machine and TrueNAS. Those are just the tools I like to use. The TrueNAS Scale ISO will install qemu-guest-agent, so you don’t need to worry about drivers. Make sure to build it with Virtio SCSI Single disk controllers. Use one 50gb OS disk for the install. Add huge data disk(s) after the install.
Promox Disk options … SSD emulation, Discard, IO Thread, No cache … and I use Write Back (unsafe). Use the Virtio NIC.

And try it again. Hopefully faster this time.