🖥️ How to Set Up a VNC Server with x11vnc — From LAN to Remote Access

Okay so I read this whole thing top to bottom and I want to say up front, the testing order at the end is the best part. Test local, then your home network, then the firewall, then remote. That is the exact thing I wish somebody had beaten into my skull two years ago when I was flailing around at 1am wondering why nothing worked.

But I want to push back on the shape of the guide. I think it teaches the 2012 version of remote access, and I want to show you why using my own setup. I am going to keep this readable for folks who are not deep in this stuff yet, because I was that person very recently.

Quick disclaimer. I am a student. Mid degree, lab in my closet, and I break it about once a week. Everything below is stuff I measured on my own gear. Not stuff I read somewhere. If I got something wrong, come at me, that is why I am posting.

Also I am borrowing the diagram format because it is honestly the best part of the writeup.

First, what port forwarding actually is

For anybody new here. Your router is the gate at the end of the driveway. Anything you go out and get comes back through just fine. But nobody coming down the road gets past that gate unless you specifically tell it to let them in.

Port forwarding is you telling the gate “if somebody pulls up asking for door number 5900, let them through and point them at the computer in the office.” That is how this guide makes your desktop reachable from off the property.

Works great when your place has an address people can actually drive to. Mine does not.

CGNAT is not a footnote. CGNAT is my internet.

The guide puts CGNAT at step 19, tucked under “hmm, what if port forwarding does not work.” I want to move it way up front, because for a lot of us it is not a maybe. It is where we start.

Here is my situation. I live on a county road in rural Texas. Nobody ran fiber out here and nobody is going to. So my internet comes over 5G, same as a whole lot of people out in the country now.

What that means in plain terms is my provider never gave my house its own address on the internet. Think of every customer on the tower sharing one mailbox out at the highway. I can drop letters in that box all day long and they go where they are supposed to go. But if you wanted to mail something back to ME specifically, there is nothing to write on the envelope. There is no box number. The carrier would have no idea which of us it was for, so it just does not happen.

                THE INTERNET
                     |
                     v
        +----------------------------+
        |   ONE SHARED MAILBOX       |   the real address on the
        |   OUT AT THE HIGHWAY       |   internet lives up HERE
        |   (me and everybody else   |   and it is not mine
        |    on this tower)          |
        +-------------+--------------+
                      |
                      v
        +----------------------------+
        |      MY HOME GATEWAY       |
        |                            |
        |  [ Port Forwarding ] <---- |   this menu exists
        |     and it does nothing    |   it is decoration
        +-------------+--------------+
                      |
                      v
                 my home network


   somebody tries to reach my place from the internet:

        internet --X   it never even gets to my gate

So I cannot port forward. Not “I have not figured it out.” I physically cannot. The setting sits right there in the menu and it is a button that goes nowhere.

And I am not some weird edge case. Rural 5G, fixed wireless, satellite, a pile of fiber resellers, giant chunks of the world outside the US. All built this way. So when a guide says “forward the port, and if that fails here is a backup plan,” a big chunk of the readers are about to lose two hours in a router menu that was never going to work for them.

Step 18 has the same problem. That section solves “my address on the internet keeps changing.” I do not have one at all. It is fixing a problem I am not allowed to have.

“A password alone should not be your whole strategy” is way too polite

The guide says it. Says it nicely. Let me put real numbers on it, because I have them.

I run a couple of servers at a cloud provider that DO have their own address on the internet, with a remote login door open on purpose. I wanted to watch what happens to a machine that just sits out there in the open.

   MY MACHINE OUT ON THE INTERNET     MY MACHINES ON THE PRIVATE NET
   ------------------------------     ------------------------------

        the whole internet            the outside world cannot
              |                       even see these exist
              | 95 to 99 break in            |
              | attempts EVERY DAY           |
              v                              v
        the login door                 a guest list check
              |                              |
              v                              v
        blocked, over and over         only my own devices get in
        automatically, all day


   that box is IDLE. nobody knows it exists. I never told a soul.
   it got found in HOURS and it has never stopped getting hit since.

Ninety five to ninety nine a day. On a boring machine nobody has ever heard of. Forever. That is just what happens the second something has an address on the internet. Automated scanners find it and they do not stop.

Now think about what it means to put a remote DESKTOP door out there instead of a text login.

   they break the remote login   ->  they get a command prompt

   they break the remote DESKTOP ->  they are sitting in your chair
                                     your browser, already logged in
                                     your saved passwords
                                     your email, your bank tab
                                     everything you left open

The guide calls that “simple, but not my preferred configuration.” I want to go harder. On a home connection it is a liability with a countdown timer on it.

I can prove the difference instead of arguing about it

Here is the part I actually wanted to share. At some point I got tired of guessing so I ran a vulnerability scanner at my own machines. That is just a tool that pokes at a computer from the outside and reports back every weakness it finds.

Same three servers. Same scanner. The only thing I changed was where I scanned FROM.

 POSITION 1  -  scanner sitting on the same home network
 ---------------------------------------------------------------
   it can see roughly a dozen open doors on each machine
   RESULT:  6 minor issues found


 POSITION 2  -  scanner coming in through my private network
 ---------------------------------------------------------------
   it can see exactly ONE door, and only after proving who it is
   RESULT:  ZERO issues. at any severity.
            it scanned for 103 minutes and found nothing.


 POSITION 3  -  scanner stuck in a walled off part of my network
 ---------------------------------------------------------------
   the firewall said no
   RESULT:  zero results in 91 seconds
            that is not a failed scan. that is the wall working.

Same machines all three times. One variable. The only thing that changed is how much the network lets a stranger touch before they prove who they are.

That is the whole argument. Port forwarding can never make it, because opening things up is its entire job.

Okay so what do I actually run

Instead of putting a door out on the internet, I built a private network that follows my devices around. Phone, laptop, the servers in my closet, even machines at a cloud provider. They all talk to each other directly and encrypted, and the outside world cannot see any of it.

                    +------------------------+
                    |   MY PRIVATE NETWORK   |
                    |   encrypted, invisible |
                    |   from the outside     |
                    +-----------+------------+
                                |
     +---------+--------+-------+------+--------+---------+
     |         |        |              |        |         |
   phone    laptop   big server    lab servers storage  cloud
                                                        servers

   every device gets a private address and a name
   that never changes on me

   zero port forwards. zero doors on the internet.

Three things I like about it.

Gear that cannot join still comes along. Some stuff is too dumb or too locked down to install anything on. A printer. A storage box. So one machine volunteers to speak for the whole house.

   the dumb gear that cannot join
   (printer, storage box, switch controls)
             |
             | plain old home network
             v
   +----------------------+
   |  one machine that    |   it announces
   |  CAN join the        |   "hey, I can reach
   |  private network     |    everything at my house"
   +----------+-----------+
              |
      MY PRIVATE NETWORK
              |
              v
        me, on my phone, 900 miles from my closet

Access is tied to ME, not to a pile of saved key files. Old way, you copy a key onto every machine, and if you lose a laptop you go hunting for every copy of it. My way, my device is on a list. Lose the laptop, pull it off the list. One click and it is gone everywhere at once.

My cloud servers and my closet are on the same network now.

   MY CLOSET                                A CLOUD PROVIDER
   ---------                                ----------------
   big server                               two servers
   lab servers     <==== PRIVATE NET ====>  (a continent away)
   storage
   monitoring

   they trade data like they are sitting on the same shelf
   no tunnel to configure. no fixed addresses. nothing exposed.

I still feel like I am getting away with something every time I look at that.

Where the guide is right, and what I keep from it

I am not against VNC at all. There are two things my private network does not solve, so here is my real setup in three tiers.

 TIER 1  -  daily driver, 99% of the time
 ---------------------------------------------------------------
   me ---> private network ---> text based remote login ---> server


 TIER 2  -  I need to actually SEE the desktop
 ---------------------------------------------------------------
   me ---> private network ---> VNC
                                ^
                                set so it only ever answers
                                the machine itself. it is not
                                listening to the network at all.


 TIER 3  -  the machine is BROKEN and tiers 1 and 2 are gone
 ---------------------------------------------------------------
   me ---> the host it runs on ---> a service door built into the
                                    virtualization software
                                    ^
                                    needs NO working networking
                                    inside that machine

Tier 2 is step 14 of the guide and I think step 14 is exactly right. Tell VNC to only answer local requests, then reach it over the private network instead of hand building a tunnel. One line change, keep it.

Tier 3 is the one nobody writes about and it is the one that got me. If a machine’s networking is wrecked, the private network dies right along with it, because it is riding on top of the regular networking. Ask me how I know.

   what I expected                  what actually happened
   ---------------                  ----------------------
   normal network broken            normal network broken
        |                                |
        v                                v
   fall back to the                 private network ALSO dead
   private network                  because it was riding on
        |                           the thing that just died
        v                                |
   I fix it, I am a genius               v
                                   fully stranded machine and
                                   me staring at a dead terminal
                                   at midnight

What saved it was that service door built into the virtualization software. It does not touch the network at all, so it worked when nothing else did. That is why tier 3 exists.

You asked what we prefer, so here is my honest answer

You listed the options at the end. x11vnc, TigerVNC, RustDesk, SSH plus VNC, WireGuard or Tailscale, something else.

My answer is your last two, stacked together.

Private network as the transport. Text based remote login on top of it for almost everything. And x11vnc locked to local only for the times I actually need to see a screen. I have not touched RustDesk, and I am honestly curious whether anybody here likes it, because it is the one thing on your list I have zero hands on time with.

Here is my stack in your format:

   YOUR REMOTE ACCESS STACK        MY REMOTE ACCESS STACK
   ------------------------        ----------------------
   the desktop                     the desktop
        |                               |
      x11vnc                          x11vnc
        |                          (answers only itself)
     port 5900                          |
        |                          private encrypted network
   port forward / SSH tunnel            |
        |                          a check that I am me
    the internet                        |
        |                               |
      my phone                       my phone

Same shape. I just swapped the middle out. Where your guide has a door on the gate, I have a private network with a guest list, and the door on the gate does not exist at all.

And your last point is the best one in the whole guide

You said this is great for old computers that are not worth carrying around but still make perfectly good home servers. I want to co sign that hard.

The main box in my closet is a decade old workstation somebody else gave up on. Two ancient server processors, a pile of memory nobody wanted, and it happily runs my entire lab. That machine is the whole reason I have anything to write about. I would never haul it anywhere. I do not have to. I get to it from my phone.

That is the actual magic here and I think you buried it in the last paragraph. Old hardware plus remote access equals a lab you can actually afford. That should be the pitch at the TOP of the guide.

Tradeoffs, because I am not selling you anything

  • The easy versions of the private network approach mean trusting an outside company to help your devices find each other. You can run that piece yourself. Plain WireGuard is the do it all yourself version.
  • If you only ever use this inside your own house and never from outside, your original setup is fine and it is simpler. I mean that.
  • Moving the login to a different door number cuts log noise. It is not security. You already said that and you are right.

What I would change about the guide

I would move the private network from #20 up to #2. Keep step 14, the local only VNC part, exactly as written, because it pairs perfectly. Push port forwarding to the back as the way we used to do it, with a note that a lot of people cannot do it at all anymore.

But the real question for the thread. How many of you can even port forward these days? Go look at the address your router thinks it has on the internet. If it starts with 100.64, sections 11 through 18 were never written for you.