If you are running into this blog post, chances are you have a Linux based server that is experiencing some kind of network issue. While not intentional, sometimes network configurations can occur that result in an undesirable outcome.

It can be from something as simple as a pesky little syntax error, a firewall rule that is perhaps too effective at blocking traffic (or a misconfigured firewall rule), or in rare instances – a temperamental cable that once swapped, makes the server accessible again.

Any of these scenarios are very fixable, but one thing is for certain – troubleshooting is the way to determine the root cause.

Helpful tips on troubleshooting network related issues on Linux servers:

Since the server is normally no longer accessible via SSH, you’ll want to become familiar with out of band console access: if you are on a Dedicated Server, you’ll want to access the IPMI console, and on VPS services you’ll want to access the VNC console. Both of these means are intentionally designed to give you access to diagnose issues in the event that remote SSH access is unavailable.

Step 1:

Once you’ve accessed the server via either of these methods, check to see if you can reach Google’s public DNS server by typing:

ping 8.8.8.8

When the server is unable to reach this destination, it’s a clear indicator that there’s an issue with the network configuration.

Step 2:

Now, it is time to dive deeper into the network configuration!

First, try pinging the server’s main IP and gateway through the ping command (from your local computer) – this will determine if your server has any form of network connectivity.

ping {your server’s main IP}

If you can successfully ping your server’s gateway IP, this is great news and indicates that the network routing is correct and further troubleshooting can begin!

While logged in, you can view the network routing to the server by executing the command:

route -n

This will show you how the network traffic reaches to your server, and it will demonstrate that traffic does reach the gateway IP of your server (a good thing!).

If for some reason, you find that your ping attempt to the main server IP did not work from your local computer terminal, it indicates that there is a connectivity issue on the particular interface on which the main IP is routed.

Pro-tip: You will want to ensure the network interface (for example “eth0”) is enabled – to do this and check all configured interfaces, you can run the following command:

ip addr

This command outputs all of the network interfaces configured and available on the server and provides you with one of two relative states: “UP” or “DOWN”

If you happen to find that the interface assigned the IP you are troubleshooting is showing a “DOWN” state, you can turn it on (and bring it “UP”) by issuing the following command:

ifdown {interface name} /* brings an interface down */

ifup {interface name} /* brings an interface up */

Example: ifup eth0 /* brings interface “eth0” up */

If for some reason the above command fails to work, there is a good chance that the interface you are troubleshooting is not in a state for the ifdown (and ifup) script to detect it. In such a situation, try using the force flag:

ifdown –force {interface name}

ifup {interface name}

When executing this command, it will show errors that indicate as to why the script is not running as expected (and successfully) – the most likely cause for this is a syntax error in your configuration file:

For users with CentOS and RHEL based servers, view your configuration file by issuing the command:

cat /etc/sysconfig/network-scripts/ifcfg-{interface name}

/* Replace interface name with your IP assigned interface */

If your interface is up and not showing any errors, but you are still not able to ping the interface – it is necessary to troubleshoot further. Without sounding too simplistic, please ensure that whatever IP’s you are troubleshooting are in fact the ones you see are assigned to you by RackNerd – please contact RackNerd support if you have any questions about this.

For users with Ubuntu and Debian servers, view your configuration file by issuing the following command:

cat /etc/network/interfaces

Do note that for clients who are specifically on Ubuntu and Debian servers above Ubuntu 18.04 LTS, this command is slightly different, as follows:

cat /etc/netplan/file.yml

At this point, if you are running a CentOS or RedHat based server, you should be seeing a network interface configuration that looks similar to the following:

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
HWADDR=”00:00:c3:b5:f2:af”
IPADDR=”XXXXXX”
NETMASK=”XXXXXX”
DHCPV6C=”no”
IPV6_AUTOCONF=”no”
IPV6INIT=”yes”
IPV6ADDR=”XXXXXX”
GATEWAY=”XXXXXX”
IPV6_DEFAULTGW=”XXXXXX”

Similarly, if you are on an Ubuntu based server (a version older than 18.04 LTS), your configuration file might look like this:

auto eth0
iface eth0 inet static
address 10.0.0.1
netmask 255.255.255.0
gateway 10.0.0.254

If you are on an Ubuntu based server 18.04 LTS (or newer), you’ll see output resembling this:

network:
version: 2
renderer: networkd
ethernets:
enp3s0:
addresses:
– 10.10.10.2/24
gateway4: 10.10.10.1
nameservers:
addresses: [10.10.10.1, 1.1.1.1]

It is important to disclose: Some configuration entries you see might be different, depending on whether the respective IP allocation is configured as a DHCP or static configuration.

On CentOS and RedHat based installs – this would relate to BOOTPROTO. On Ubuntu and Debian installations, this would pertain to inet static.

It is also important to note that the grub configuration file will dictate whether the device needs to be in the “UP” state during server reboots. On CentOS and RedHat installs, this will relate to ONBOOT. On Ubuntu and Debian installs, this will be depicted as auto.

Additional note (for those who run Ubuntu server versions newer than 18.04): your network configuration is managed by the systemd service (rather than by Network Manager) – in such cases, the netplan to load upon reboot is directly specified in your grub configuration file.

After changes have been made, you are now ready to restart your network service by executing the following command:

On Ubuntu and Debian installs before version 18.04 LTS:

service restart networking

On Ubuntu based servers with installs newer than 18.04 LTS:

netplan apply

On CentOS and RedHat based servers:

service restart network

Alternatively:

ifdown {interface name}
ifup {interface name}

Pro-tip: Avoid doing network restarts (as well as ifup and ifdown scripts) if you are logged into the server via SSH. If the network configuration has errors, it could (and likely will), break your current SSH session.

Additional tip: Insight into network related errors can be found in the following log files, which you can access (regardless of OS version):

/var/log/messages ( grep with the respective interface)
dmesg ( grep with the respective interface )

eg: grep eth0 /var/log/messages
grep eth0 dmesg

If the above did not work, it is time to explore and troubleshoot other possible scenarios:

COMMON SCENARIO: A misconfigured firewall rule might be causing connections to be blocked.

You can check where a connection is blocked when it comes to the server by utilizing the following tools:

On Ubuntu installs (Replace public IP with your server’s IP):

mtr {public_IP}

On Debian installs (Replace public IP with your server’s IP):

traceroute {public_IP}

On CentOS and RedHat based servers (Replace public IP with your server’s IP):

traceroute {public IP}

Alternative,

tracepath {public_IP}

These commands will pass over the network to the given destination and show the latency to each node that replied. In the event the trace does not reach the server’s IP or gateway – it can indicate a firewall is running on the server (blocking the connection).

To diagnose if this is the case, you can verify the iptables rules in Linux by issuing the command:

iptables -L -n

Sometimes the ICMP protocol is blocked by the server firewall which will deny the server from responding to ping requests. Additionally some policies can be substantially more strict and block regular connections and/or specific IP’s. To see if the issue is related to a firewall rule, you can stop the firewall (iptables, or UFW) – many times, doing so will result in you being able to connect to the server as desired (therefore the issue related to the firewall rules).

The first step to troubleshooting is to stop your firewall temporarily, to determine if it is the root cause to your issue.

To stop iptables on a CentOS or RedHat based server:

service iptables stop

To stop UFW on an Ubuntu or Debian based server:

ufw disable

After stopping the firewall, check to see if your server starts to respond to ping again. If it is, then you will want to review your firewall rules before enabling iptables or ufw again.

While in practice, the above troubleshooting methods will resolve the vast majority of the cases we see – it is possible that something else could be the cause. If at any time you encounter issues troubleshooting your server, or have exhausted these options, please do not hesitate to reach out to RackNerd support for assistance!