Inspecting Phone Network Connections

Mitmproxy setup (for macbook):

1. If not already installed, install Homebrew, see https://brew.sh/. Use:

• /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

1. Install mitmproxy, see https://mitmproxy.org/. Use:

• brew install mitmproxy

1. Setup macbook as wireless AP:

• Connect macbook to one end of a wired ethernet cable (you'll need a usb-to-ethernet adapter for this) and connect other end of ethernet cable to an internet gateway e.g. your home router. This will be used to connect the macbook to the internet.
• Open Settings, go to General, Sharing and click on "Internet Sharing".
• In the "Share your connect from" drop down box select the wired ethernet connection.
• In the "To computers using" box choose Wi-Fi.
• Click start. You should see the Wifi icon in the top bar of the macbook change to be grayed out. Your macbook is now acting as a WiFi access point. Devices connecting to it will have their traffic routed over the wired ethernet cable connected to the macbook. The name of the Wifi network will be the same as the name of the macbook, on your phone open settings, look for the available Wifi networks and connect to the macbook.

1. Setup transparent proxying using mitmproxy, see https://docs.mitmproxy.org/stable/howto-transparent/ for details. In summary:

• Download file https://raw.githubusercontent.com/doug-leith/mitm-capture-setup/main/pf.conf which contains these two lines:

rdr on bridge100 inet proto tcp to any port {80,443} -> 127.0.0.1 port 8080 block drop quick on bridge100 inet proto udp to any port 443

• Setup the firewall to redirect Wifi traffic to mitmdump using: sudo pfctl -f pf.conf; sudo pfctl -e

• Edit file /etc/sudoers and add line:

ALL ALL=NOPASSWD: /sbin/pfctl -s state

• Type mitmproxy to start up mitmproxy for the first time, then exit. This will create folder ~/.mitmproxy that contains the CA cert used by mitmproxy

• Update the permissions using chmod a+r ~/.mitmproxy/mitmproxy-ca.pem

• Now type sudo -u nobody mitmdump --mode transparent --showhost --ssl-insecure --rawtcp to startup mitmproxy. Traffic from the phone will now be routed to mitmproxy. Typically you'll see errors as the phone detects the presence of mitmproxy intercepting connections.

Mitmproxy setup (Raspberry Pi)

1. A Raspberry Pi is fine for running mitmproxy too, and much cheaper than a Macbook! But best to use a newer Raspberry Pi 4 as I've had trouble trying to use an older Pi 3. To set it up, first follow the instructions to configure Pi as a Wifi router.

2. Install mitmproxy by downloading the binary from https://mitmproxy.org/downloads/using (the version installed by apt-get install mitmproxy lags the latest release quite a bit). Setup the firewall rules to redirect traffic to mitmproxy by running this shell script.

3. Run mitmproxy using e.g. mitmdump --mode transparent --showhost --ssl-insecure --rawtcp

Install mitmproxy CA cert as trusted on phone

The next step is to install the CA cert of mitmproxy as a trusted cert on the phone.

iPhone

1. Go to Settings-WiFi and connect to laptop access point
2. Open Safari and navigate to url http://mitm.it (note, http not https). Click "Get mitmproxy-ca-cert.pem" for iOS. You'll see a message asking "This website is trying to download a configuration profile. Do you want to allow this?", choose "Allow".
3. Go to Settings-General-VPN & Device Management (Settings-General-Profile on older iOS), you should see mitmproxy, click on it and choose "install".
4. Go to Settings-General-About and at bottom of page choose "Certificate Trust Settings". You should see mitmproxy, click the toggle to enable full trust
5. As a test, go back to Safari and navigate to https://leith.ie/nothingtosee.html. You should see connections being logged by mitmdump on the laptop as you type the URL, and then the actual connection to leith.ie/nothingtosee.html. You may still see a few connections reporting errors, but mostly the connections should be accepted now - the connections that fail are being made by system processes, not Safari, and we need to jailbreak the phone and install SSL Kill Switch to make them work with mitmdump (see below).

Android

To install a trusted (or system) CA cert on Android requires rooting the phone.

1. Root your phone - typically this involves unlocking the bootloader (often the hardest task!). The details of rooting are phone dependent, but you want to get to a setup where you have access to a rooted adb shell and have Magisk https://github.com/topjohnwu/Magisk installed.

2. Download the Trustusercerts Magisk module from https://github.com/lupohan44/TrustUserCertificates or use my local copy https://raw.githubusercontent.com/doug-leith/mitm-capture-setup/main/trustusercerts-v1.2.zip. Connect the phone to a laptop by usb cable and on the laptop use:
   adb shell push trustusercerts-v1.2.zip /sdcard/Download
   to copy the module to the phone. On the phone, open the Magisk app and install the trustusercerts-v1.2.zip module from storage (it is in the phone Downloads folder).

3. Install the mitmproxy CA cert on the phone

• On the wifi access point running mitmdump, copy the file mitmproxy-ca-cert.pem from folder ~/.mitmproxy/. This is the CA cert that mitmdump uses to sign https certs when it intercepts network connections.
• Copy the mitmproxy CA cert to the phone: adb push mitmproxy-ca-cert.pem /sdcard/Download
• On the phone install the mitmproxy CA cert as a client CA cert. The details change from phone to phone, but tyically open the Settings app, then the Security screen, scroll down and there is usually an "Encryptions" or "Additional Security Settings" button, click that. Then select "Install CA cert", skip past the warnings, and select the mitmproxy-ca-cert.pem file from the phone Downloads folder.
• Restart the phone. The Trustusercerts Magisk module should promote the client CA cert that you installed to a system cert. You can check that the mitmproxy cert has been successfully installed by looking at Settings-Security-Encryptions and credentials-Trusted Credentials on the phone - that lists the trusted CA certs and mitmproxy should be listed there if things worked ok.

Note: installing the mitmproxy cert as a system cert is enough to allow the traffic generated by Google Play Services etc to be decrypted by mitmproxy, but other apps may include additional pinning of SSL certs. Those need to be treated on a case by case basis, and typically require use of a custom Frida script to bypass the checks.

iPhone jailbreak setup (palera1n + Sileo)

To bypass SSL cert checks made by iPhone system processes it's necessary to jailbreak the phone. These steps use palera1n (rootless) with the Sileo package manager and the ElleKit tweak injector. checkra1n and Cydia are no longer maintained.

palera1n uses the checkm8 bootrom exploit, so it works only on A8-A11 devices (e.g. up to iPhone 8 / iPhone X) running iOS 15 or later. On A11 devices (iPhone 8 / X) running iOS 16 or later, the phone must have no passcode set. palera1n is semi-tethered: after every reboot the phone starts un-jailbroken and you need to run palera1n again.

1. Download palera1n for macOS from https://github.com/palera1n/palera1n/releases (I used v2.2.1) and make it executable, e.g. chmod a+x palera1n-macos-universal.
2. Connect the phone to the Mac by USB. Use a USB-A to Lightning cable: USB-C connections, especially on Apple silicon Macs, often fail to enter DFU mode, so I use an older Mac with a USB-A port for jailbreaking. Run ./palera1n-macos-universal -l (-l selects a rootless jailbreak) and follow the on-screen instructions to put the phone into DFU mode.
3. Once the phone has rebooted, connect it to a Wifi network (a normal network, not the mitmproxy one), open the palera1n loader app on the phone and install Sileo. When asked, set a password; it is used for sudo and for ssh as user mobile.
4. Open Sileo and install:

• openssh-server (ssh access), ellekit (tweak injection) and PreferenceLoader (settings pane for tweaks);
• optional: rsync; and Frida, after adding the source https://build.frida.re, see https://frida.re/docs/ios/.

1. Install my modified version of SSL Kill Switch 3, which also hooks Apple system processes and still lets the push notification daemon (apsd) connect. Download the prebuilt package ssl-kill-switch3-DL.deb (version 1.1-6), or build it from the source in folder ssl-kill-switch3-DL with Theos (make package ROOTLESS=1). Then copy it to the phone and install it:

• scp ssl-kill-switch3-DL.deb mobile@<phone IP>:/var/jb/var/mobile/
• ssh mobile@<phone IP>
• sudo dpkg -i /var/jb/var/mobile/ssl-kill-switch3-DL.deb
• sudo killall -9 SpringBoard (respring, so that the new settings entry appears; close the Settings app first if it is open)

1. On the phone open Settings; at the bottom there is now an entry "SSL Kill Switch 3 DL". Open it and enable "Disable Certificate Validation". The tweak is loaded when a process starts, so reboot and jailbreak again with palera1n -l (step 2) so that all system processes pick it up.
2. Change the Wifi settings to connect to the mitmproxy access point and install the mitmproxy CA cert as described above. You should now see connections by system processes being successfully decoded by mitmproxy. You may see some warning messages about self-signed certs (these are Apple certs), but the --ssl-insecure option to mitmdump allows these to pass.

Notes:

1. To access the iPhone from the command line over USB, install libimobiledevice using brew install libimobiledevice (see https://libimobiledevice.org/). Then use:

iproxy 2222 22 &

ssh -p 2222 mobile@localhost

and use sudo for root. The password is the one you set when installing Sileo. To copy files off of the phone use:

• scp -P 2222 mobile@localhost:<file> .

On a rootless jailbreak, jailbreak files (packages, tweaks, /var/jb/usr/bin tools) live under /var/jb.

1. Once libimobiledevice is installed, to get device details use:

• ideviceinfo

Huawei Setup

Rooting a Huawei handset is more involved than for other handsets since Huawei no longer officially support bootloader unlocking. Extra steps are also needed to unpin SSL cert checks made by system processes. See here for more details

Capturing traffic

Saving a capture to a file

Add -w <file> to the mitmdump command used above. mitmdump then saves every connection to that file as well as printing it:

• macOS: sudo -u nobody mitmdump --mode transparent --showhost --ssl-insecure --rawtcp -w /tmp/capture.mitm. When mitmdump runs as user nobody it can only write to a folder that nobody can write to, such as /tmp.
• Raspberry Pi: mitmdump --mode transparent --showhost --ssl-insecure --rawtcp -w capture.mitm

Stop mitmdump with Ctrl-C, not kill -9. A connection is only written to the file when it closes, and Ctrl-C writes out any connections that are still open.

Reading a capture with mitmdump

To list the connections in a saved capture, use:

• mitmdump -nr capture.mitm --showhost

-r reads the file and -n stops mitmdump from starting a proxy. Add --flow-detail 2 to also show the HTTP headers, --flow-detail 3 to also show the message bodies (truncated), or --flow-detail 4 to show them in full. To select connections, add a filter expression, e.g. mitmdump -nr capture.mitm --showhost --flow-detail 3 "~d apple.com". To browse a capture interactively, use mitmproxy -r capture.mitm.

Most of the traffic sent by phone system processes is binary protobuf, which mitmdump shows only as raw bytes. To decode it you need the message schemas, see below.

Decoding iPhone traffic

ios-protobuf-decoding contains ios_decoder, which decodes a saved capture offline using schemas recovered from Apple's binaries. It needs Python 3.12 or later and protoc 27 or later (brew install protobuf):

git clone https://github.com/doug-leith/ios-protobuf-decoding.git
cd ios-protobuf-decoding
python3 -m venv venv
./venv/bin/pip install -r requirements.txt
./venv/bin/python3 -m ios_decoder.report capture.mitm

By default every connection is listed, with bodies that could not be decoded shown as a short placeholder or a preview of the raw bytes. Add --decoded-only to show only the connections where something was decoded, --user-agent geod to select one process, or --url <regex> to select endpoints. Its README lists all the options.

Decoding Android traffic

android-protobuf-decoding has protobuf schemas for Google Play Services traffic (Clearcut logger, Firebase Analytics, checkin, Play Store and others). It also has a mitmproxy addon, decoding_helpers.py, that decodes a saved capture:

git clone https://github.com/doug-leith/android-protobuf-decoding.git
cd android-protobuf-decoding
python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt
mitmdump --flow-detail 0 -s decoding_helpers.py -nr capture.mitm

Apple push notifications (APNs) on iPhone

An iPhone keeps a long-lived connection to the Apple Push Notification service at courier.push.apple.com, on TCP port 5223 (falling back to 443). The setup above doesn't capture this connection. Capturing it needs two extra steps:

1. Redirect port 5223 to mitmdump as well.

• On macOS, change the first line of pf.conf to rdr on bridge100 inet proto tcp to any port {80,443,5223} -> 127.0.0.1 port 8080 and reload it with sudo pfctl -f pf.conf.
• On a Raspberry Pi, use sudo iptables -t nat -A PREROUTING -i wlan0 -s <phone IP> -p tcp --dport 5223 -j REDIRECT --to-port 8080.

1. Load the APNs addon apns_mitm.py from ios-apns-decoding when capturing:

• mitmdump --mode transparent --showhost --ssl-insecure --rawtcp -w capture.mitm -s apns_mitm.py

Without the addon the TLS handshake fails. The addon also does two other things:

• It raises mitmproxy's idle timeout to one hour. The default of 600 seconds closes the connection just before the phone's first keep-alive, which comes after 10 minutes.
• It appends each decoded message to apns_decoded.jsonl as it arrives. The connection stays open for hours and is only written to capture.mitm when it closes, so this log is the copy that survives if mitmdump stops abruptly.

To decode the APNs messages, run ./venv/bin/python3 apns_mitm.py capture.mitm in the ios-apns-decoding folder. To decode from the log instead, use ./venv/bin/python3 apns_mitm.py --jsonl apns_decoded.jsonl. ios_decoder (above) also decodes APNs connections, together with the rest of the traffic in a capture.