Troubleshooting connectivity issues

Single user or small group connectivity issues can be resolved by confirming that the networking functions required for a user workstation to access local and remote network resources are functioning properly. The basic requirements or items to confirm include:

• Enabling the correct network interface(s) (workstation configuration)
• Confirming layer 1 (physical) connectivity
• Obtaining an IP address, subnet mask, and default gateway for each interface (DHCP)
• Obtaining the MAC address of the default gateway or other local network services (ARP)
• Obtaining the IP address of a network service (DNS)
• Connecting to a network service (TCP handshake or UDP response)

We'll briefly discuss each of these in order; while the first two steps will not involve using Wireshark, they are a necessary part in a troubleshooting approach. If the connectivity issue is affecting a group of users or a whole office, the first step is probably not applicable.

Enabling network interfaces

While it may seem obvious that network interfaces need to be enabled, the assumption that they are automatically enabled (especially for the wireless connectivity) by default upon device boot up may be false.

On Windows, you can use the command-line utility ipconfig to view the status and basic configuration (IP address, subnet mask, and default gateway) of network interfaces; on Linux or MAC devices, the equivalent command is ifconfig or ip.

Confirming physical connectivity

If a connectivity problem is isolated to a single user's workstation, the physical connections are suspected. There are a few items to check, and the troubleshooting steps that can be taken are as follows:

• If there is a problem with the Ethernet cable from the workstation to a wall jack, you need to swap the cable with a different one.
• If there is a problem with the cabling from the user's wall jack to the switch port, you need to temporarily plug the user's Ethernet cable into another (known good) wall jack.
• If there is a problem with the switch, switch port, or port configuration, you need to temporarily plug the user's port cable into another (known good) port. Be aware that some network security policies call to disable switch ports until they are needed or configuring the port to be associated with a single, specific MAC address. If so, a port may not work when you plug into it although there is nothing physically wrong with it.

Obtaining the workstation IP configuration

Unless the workstation was manually configured, it will need to get its IP address, subnet mask, default gateway, and DNS server settings from a DHCP server. If this does not appear to be working properly (after checking the configuration using ipconfig (Windows) or ifconfig, (Linux or Mac OS X)), you need to perform a packet capture during the workstation initialization/boot-up process using a TAP or SPAN port and investigate the DHCP requests and responses.

There are eight DHCP message types (not to be confused with the two Bootstrap Protocol types, Boot Request and Boot Reply):

For a workstation that is booting up and was previously working on the network, you'll generally see the DHCP Request and Acknowledgment packets verifying that the workstation can still use a previously leased address. On an entirely cold start up, the first two DHCP packets will be DHCP Discover and DHCP Offer packets, followed by the Request and ACK packets.

In a DHCPv6 environment, the typical packet sequence is DHCPv6 Solicit, DHCPv6 Advertise, DHCPv6 Request, and DHCPv6 Reply.

The fields to verify in a DHCP Response packet (or similar fields in a DHCPv6 Advertise packet) include the following four fields:

• Your (client) IP Address: This is the offered IP address for this workstation
• Subnet Mask: This is the subnet mask to use on this network
• Domain Name Server: This is the DNS server IP address
• Router: This is the IP address of the default gateway to use

This is minimum data required for any network communications; an example of these fields being provided in a DHCP Reply packet is illustrated in the following screenshot:

You can apply Wireshark display filters to isolate DHCP packets; the filter is bootp, as this is the legacy name for DHCP:

• DHCP display filter: bootp bootp.option.dhcp == 5 (DHCP Message Type 'ACK')
• DHCPv6 display filter: dhcpv6 dhcpv6.msgtype == 2 (DHCPv6 'Advertise')

You can save the basic bootp and dhcpv6 display filters as a Filter Expression Button (FEB) after entering the filter string in the textbox on the Display Filter toolbar, clicking on Save, and giving the button a name such as DHCP Pkts and DHCPv6 Pkts respectively. Alternatively, you could combine both filters with an or (||) in one button, as shown in the following screenshot:

You might want to save another FEB that displays an abnormal DHCP condition packets using the following display filter string and call the DHCP Errors button or a similar as follows:

bootp.option.dhcp == 4 || bootp.option.dhcp == 6 || bootp.option.dhcp == 7

Similar abnormal event display filters for DHCPv6 could include:

dhcpv6.msgtype == 8 || dhcpv6.msgtype == 9 || dhcpv6.msgtype == 10

You can research more about DHCP, DHCPv6, and the various DHCPv6 message types online or from other sources if you need to analyze these in more detail.

Obtaining MAC addresses

A workstation will utilize the ARP protocol to obtain a MAC address for known IP addresses of network services, such as its default gateway or the DNS server if it's located on the same network segment. The ARP protocol and how it typically functions has already been covered in Chapter 2, Networking for Packet Analysts.

You may want to create an ARP FEB using the arp display filter syntax to make it quick and easy to inspect those packets.

Obtaining network service IP addresses

A client workstation sends queries to a DNS server to obtain an IP address for a given hostname; the DNS server responds with the information or asks other DNS servers for the information on behalf of the client.

The format of the DNS query and response packet fields as displayed in the Wireshark Packet Details pane is fairly intuitive. An example of a DNS response packet containing a resolved IP address for time.windows.com, which actually provided the IP address (137.170.185.211) for the alias time.microsoft.akadns.com is shown in the following screenshot:

If a client workstation cannot obtain the IP address of a web service or application server, a packet-level investigation of the request (which URL or hostname is being requested), and what the response is from the DNS server (if any) should be revealing. A comparison of a failing query with queries that work properly for other hostnames or from other workstations should reveal the root of the problem (if DNS is the problem). Failure to obtain an IP address can be caused by an inoperable DNS server, improper hostname or URL, or a problem with connectivity from the user to other parts of the network, which we'll check next.

Basic network connectivity

A few simple tests can confirm that basic network connectivity is working, or reveal a routing issue or another issue that needs to be addressed by the network support team.

Capturing and analyzing the ICMP packets sent and received during the following tests can be revealing; although, the test results themselves are often telling enough:

• Ping the user's default gateway using the default gateway IP address obtained from using ipconfig /all (Windows) or ip addr show (Linux) to confirm that the user workstation has basic connectivity on the local network.
• Ping the hostname or URL of the target server. If this fails (request timed out message), try to ping other hosts or URLs. If necessary, inspect the DNS and/or ICMP responses in a packet capture of these tests to determine the nature of the failure. Otherwise, take note of the average round trip times.
• If a ping works to the default gateway but pinging other targets fails, a traceroute to a target server can reveal where in the network path connectivity ceases to function or is blocked.

  Note

  The traceroute command-line utility in Windows is tracert, whereas for traceroutes on Linux/Unix and Mac OS X machines, the command is traceroute. To do a traceroute in Windows, open a Command Prompt (CMD) window and type tracert <hostname or IP Address of target>. In most other environments, open a terminal window and type traceroute <hostname or IP address of target>.

If you can ping the target server and network connectivity is functioning, you can move on to the next step in the troubleshooting process. If not, be aware that some hosts may be configured to not respond to ICMP ping requests, and/or ICMP is blocked by a firewall between the user and server for security reasons. So, the inability to ping a device is not necessarily a sign of a network problem. Traceroute results should help determine how far and to what extent network connectivity is functioning in the path towards the target server; testing to other targets should be revealing as well.

An example of pinging a default gateway, then a URL, and finally performing a traceroute to the target URL is depicted in the following screenshot:

Connecting to the application services

If network connectivity from a user workstation to a target server is functional (as proven by the ability to ping the host), a problem connecting to a specific application hosted on that server may be caused by a number of factors:

• The URL or port used by the client to access the application is wrong
• The port used to access the application is blocked by a firewall
• The application service is not turned up or is not working properly

The first of these factors is far more likely for a single user issue. Any of the last two factors would prevent anyone in a group or the whole organization from accessing the application. A packet-level analysis (from the client side) of a user attempting to connect to an application that is blocked should result in ICMP messages: Destination Host is Unreachable or Destination Port is Unreachable, or there will be no response at all if ICMP messages are being blocked by a firewall.

If the server is up, the application is reportedly operational but cannot be accessed; a client-side capture does not offer any solid clues, but a packet capture of the TCP session setup (if any) from or near the server end should be revealing.