Mitmproxy 4

We’ve just released mitmproxy v4.0, and it’s an absolute corker. Among the usual long list of bugfixes and improvements, one thing stands out: speed. Users should see about 4x improvement in core request throughput for mitmdump, and a 10x or more improvement for mitmproxy console. Let’s dig into the details.

Speed!

At its core, mitmproxy co-ordinates communication by shunting messages between client connection threads and a single main controller thread. The mechanics of this process has always been a drag on our performance, especially when interactive event loops enter the picture. This release shifts the core controller thread to Python’s built-in asyncio event loop. The result is rather remarkable: a roughly 4x speedup in mitmdump, and a more than 10x speedup for mitmproxy console.

Console key binding configuration

The key bindings for mitmproxy console can now be configured through the keys.yaml file, under the mitmproxy configuration directory (~/.mitmproxy). Here’s an example of this file’s syntax:

-
  # Simple global binding
  key: ctrl a
  cmd: replay.client @marked
-
  # Bind key only in the flowlist
  key: "1"
  ctx: ["flowlist"]
  cmd: console.nav.down
  help: Go to next flow

Please see the docs for more information.

Other notable changes

There are a number of other changes to note:

• Mitmproxy now only supports Python 3.6 and newer. The immediate reason for this shift is improvements to the the asyncio module in 3.6. We plan to be fairly aggressive about deprecating support for older versions of Python from here on.
• The --conf and --cadir command-line flags have been removed, and replaced with --confdir. This flag directs mitmproxy to use a specified configuration directory, with all the configuration files it might contain.
• The allow_remote option has been replaced by the much more flexible block_global and block_private options.
• We no longer magically capture print statements in addons and turn them into logs. Please use the ctx.log.* functions explicitly.

Release cadence

This release sees a shift in the mitmproxy project’s release cadence. It’s been just less than three months since the release of v3.0, and we intend to maintain this pace into the future. Henceforth, you should expect mitmproxy releases at approximately 2-3 month intervals.

GSoC

We’re also very happy to announce that the project will host two brilliant students under the Google Summer of Code 2018 program this year. GSoC has played an important role in mitmproxy’s history - it’s an opportunity for us to induct talented co-conspirators into the project as full contributors.

Roman Samoilenko (@kajojify) will be working on the new command language for mitmproxy. This is the glue that binds together mitmproxy and its addons through a system of flexible, typed commands. Once this is in place, it will make entirely new patterns of interaction with mitmproxy possible.

Pietro Tirenna (@madt1m) will be modernising mitmproxy’s serialisation format. The new format will allow random access and in-place modification of flows and provide APIs for storing auxiliary flow data. This is a critical change that will set the stage for a huge slew of improvements to mitmproxy in coming releases.

We’d like to thank Roman and Pietro for tackling two very ambitious and important projects with us.