Intranet penetration is a tool that is often used in development work. Let’s look at the definition of this tool:

Intranet penetration is NAT penetration, a term used for network connection. When a computer is in a LAN, the computer nodes on the Intranet and the Internet need to be connected for communication, and sometimes Intranet penetration is not supported. It means mapping ports, allowing computers on the outside to find computers on the inside

In short, a machine that accesses the Intranet using a fixed external IP address. There are many Intranet penetration solutions, such as peanut shell, FRP, NGROk, Shootback, etc. If you want a stable and high performance penetration product, it is recommended to pay for peanut shell. If it is used for testing and development, then some open source products can also be considered.

This article briefly describes how to write a simple Intranet penetration tool in Elixir, where all the project code resides.

WHY

Q: Why reinvent the wheel when there are already so many powerful penetrating tools on the market?

Answer: 1. 2. While most of the solutions on the market are large and versatile, this wheel pursues small and beautiful.

Q: Why did you choose Elixir

Answer: 1. Elixir is known as the best choice of socket programming, socket programming experience nice; 2. Elixir is the Erlang virtual machine language, oriented to concurrent programming, eliminating many implementation details of multiplexing.

PREPARE

To understand this article, you need the following prerequisites:

  1. Computer network foundation;
  2. Elixir foundation or Erlang Foundation;

HOW

STEP 1: Build a project skeleton

First, make clear the physical form of the software in mind. Intranet penetration is a typical CS architecture, so there must be clients and servers:

The general physical form is shown as follows:

  • Deployed on the Intranet, the client connects to Intranet applications and forwards the traffic from Intranet applications to the server. In addition, the client receives the traffic from the server and forwards the traffic to the Intranet applications.
  • The server is deployed on a machine that can be directly connected to the network. It receives both the traffic from the client and the external traffic, forwards the external traffic to the client, and forwards the client traffic to the external connection.
  • A TCP connection is maintained between the client and the server. If multiple penetrating applications want to reuse the connection between the client and the server, they need to design a protocol to distinguish the traffic belonging in a single connection.

Two projects will be established:

Mix New Tunnel_EX -- Umbrella # Project top layer, designed as an umbrella project, otherwise, Tunnel_ex /apps mix new client --sup # client mix new server --sup # server mix new Commmon Helpers or somethingCopy the code

So far the project looks like this:

. ├ ─ ─ apps │ ├ ─ ─ client │ │ ├ ─ ─ the config │ │ ├ ─ ─ lib │ │ ├ ─ ─ mix. Exs │ │ ├ ─ ─ mix. Lock │ │ ├ ─ ─ the README. Md │ │ └ ─ ─ the test │ ├ ─ ─ Common │ │ ├ ─ ─ lib │ │ ├ ─ ─ mix. Exs │ │ ├ ─ ─ the README. Md │ │ └ ─ ─ the test │ └ ─ ─ server │ ├ ─ ─ the config │ ├ ─ ─ lib │ ├ ─ ─ mix. Exs │ ├ ─ ─ mix. Lock │ ├ ─ ─ the README. Md │ └ ─ ─ the test ├ ─ ─ the config │ └ ─ ─ config. Exs ├ ─ ─ LICENSE ├ ─ ─ mix. Exs ├ ─ ─ mix. Lock └ ─ ─ the README, mdCopy the code

After compiling the code, you only need to compile and package it in the client and server directories respectively.

STEP 2: Agreement

In the practice of CS programming, the most important thing is the design of protocol, good protocol can save resources and improve performance. Of course, the protocol design of this project need not be too rigorous, after all, it is only an experimental project. First let’s consider the objectives of the protocol:

  1. Replay the behavior of external traffic on the server side from the client to the internal application. For example, if the external server establishes a TCP connection with a certain port monitored by the server, the client immediately establishes a TCP connection with the corresponding internal service. The external traffic to the server port, the client must also be intact to the internal application;
  2. Because the connection between client and server needs to be reused, the protocol must be able to distinguish between traffic of different penetration combinations. For example, the traffic received by port 8080 can only be forwarded to port 80, but not to port 81 or 82.

To accomplish this, I devised a simple protocol:

1. Establish a connection between the client and the server

The penetration we designed can be one server corresponding to multiple clients, so clients should report their relevant information when initiating connections. We can design as follows:

In connection, the client by sending its own IP address (the IP address is not necessarily true IP, as long as you don’t collide with the other client IP address can, in fact, here is a client id, use the IP address can also be physical meaning) to the server, the server will store an IP address corresponding to the mapping relationship of the socket The format is as follows:

| 0x09 | 0x01 | ip0 | ip1 | ip2 | ip3 |
Copy the code

The first two bytes are fixed to <<9, 1>>, followed by four bytes of IP address. Once the server receives the packet in this format, it automatically knows that the packet is reported by the CLIENT’S IP address. In this case, the server sends an acknowledgement to the client indicating that it has received the IP address report in the following format:

| 0x09 | 0x02 |
Copy the code

The value is <<9, 2>>. If the client does not receive the reply receipt within a period of time, it indicates that the server is suspended. This part of the exception handling is complicated, I did not consider this in the first version, programming should focus on the main flow first.

2. Establish a TCP connection with the server

When external traffic establishes a TCP connection with a port monitored by the server, the server should notify the client of this event. We do the following design:

| 0x09 | 0x03 | key::16 | client_port::16 |
Copy the code

A fixed 2-byte <<9, 3>> is followed by a 2-byte key and a 2-byte port number. Where key represents the ID of an external connection and client_port represents the internal port to which the packet is sent. When an external connection is established with the server, a 2-byte key is assigned to each TCP connection and the mapping between the key and the connection is stored. When it is forwarded to the client, the client establishes a connection with the internal application based on the client_port and stores the mapping between the key and the internal connection. In this way, we can distinguish between server and client communication packets by key.

Need special attention is, external and server-side interaction is unable to perceive the client side of the situation, it is likely that the external completion of TCP connections, immediately start sending traffic, but from the server to the client’s network packet is not necessarily in accordance with the order, so the client successfully establish a TCP connection with internal application before, the service side is best to external flow caching, Wait for the client to receive a signal indicating that a connection is successfully established, and then send the cache traffic to the client in sequence. For simplicity, the receipt is designed as follows:

| 0x09 | 0x03 | key::16 |
Copy the code

3. Communication phase

When the TCP connection is established between the server and the client, the data flow of the application layer is sent. We do the following design:

| key::16 | real packet |
Copy the code

It is easy to understand, based on the key to find the corresponding internal connection, and then forward the real traffic.

4. The connection is closed

When the external connection to the server is closed, the server should also notify the client to close the internal connection. The format is as follows:

| 0x09 | 0x04 | key :: 16 |
Copy the code

It is preceded by a fixed <<9, 4>> beginning, followed by a key indicating which connection needs to be closed.

STEP 3: TALK IS CHEAP, SHOW ME THE CODE!

Next, we start coding according to the protocol.

The code skeleton

The client part is easier, so let’s start with that part. First of all, the client form should be conceived. The client needs to actively connect to both Intranet applications and the server, so the client should be composed of two clients:

  • The client of the internal application is named worker, and the process should be created dynamically.
  • For the server side, we call it selector(selector because traffic from the server is multiplexing TCP connections, and this layer of code has to do some selective distribution).

The client structure should look like this:

├── Client │ ├─ Application. Ex # = main │ ├─ ├─ Socket_store ├── ├─ ├─ ├─ ├─ ├. ├── ├─ ├─ ├Copy the code

As for the server, there are more things to consider. The server does the following:

  • Listen for client connections;
  • Listen for external connections;
  • Forwards external traffic to the client.
  • Forwards the traffic from the client to the external device.

There are two types of TCP server in the server. The structure can be as follows:

├── Server │ ├─ Application. Ex # = main │ ├─ External_worker. ex # External_worker. ex # ├── Internal_listener. ex # │ ├─ Internal_worker. ex # Socket_store.ex # ├─ ├─ garbage, ├─ garbage, ├─ garbage, ├─ garbageCopy the code

This hierarchy is rather crude in design and could be refined, but of course the first edition focuses on the main contradiction first.

Client The server establishes the connection

In the Erlang virtual machine, you can delegate a socket to an Erlang GenServer process and let GenServer delegate some of the socket’s event responses (sure enough, Erlang is the best OO model). In this project, We proxy all sockets using the GenServer process, somewhat similar to class instances in other OO languages.

So let’s look at the client selector, first of all it’s a client process that actively connects to the server port. So let’s pull up a GenServer.

defmodule Client.Selector do def start_link(opts) do {name, opts} = Keyword.pop(opts, :name, __MODULE__) GenServer.start_link(__MODULE__, opts, name: Name) end def init(_opt) do send(self(), :connect) nil}} end def handle_info(:connect, state) do {host, Logger.info("Connecting to #{host}:#{port}") with {:ok, IP} < - host | > to_charlist | > : inet. Parse_address (), # address resolution {: ok, the sock} < - : gen_tcp. Connect (IP, port, [: binary, active: true, packet: 2]), # establish connection localhost < -client_cfg (), # obtain configuration local IP address {:ok, {ip0, ip1, ip2, ip3}} <- localhost |> to_charlist |> :inet.parse_address() do # handshake :gen_tcp.send(sock, <<0x09, 0x01, ip0, ip1, {:noreply, map. put(state, :socket, sock)} else {:error, Reason} -> logger.warn ("reason -> #{inspect(reason)}") process.send_after (self(), :connect, 1000) # state} _ -> {:stop, :normal, state} end end endCopy the code

This part of the code is mainly to establish a server connection, at the same time according to the protocol to report their OWN IP address.

As per the protocol, the server will return a packet.

def handle_info({:tcp, _socket, <<0x09, 0x02>>}, state) do
    # handshake finished
    Logger.info("handshake finished")
    {:noreply, state}
end
Copy the code

In the socket proxy process, the message sent to the socket is converted into a MSG sent to the Erlang microprocess, which simplifies our programming model and saves us from having to do the corresponding event Handle on our own initiative recV.

The server listens for the client’s connection requests and responds accordingly.

The corresponding server-side code is as follows:

InternalListener do@moduledoc """ "" require Logger use GenServer alias Server.{InternalWorker} def start_link(opts) do {name, opts} = Keyword.pop(opts, :name, __MODULE__) GenServer.start_link(__MODULE__, opts, name: name) end def init(_opts) do port = server_port() {:ok, acceptor} = :gen_tcp.listen(port, [:binary, active: false, reuseaddr: true, packet: 2]) send(self(), :accept) Logger.info("Accepting connection on port #{port}..." ) {:ok, %{acceptor: acceptor}} end def handle_info(:accept, %{acceptor: Acceptor} = state) do {:ok, sock} = :gen_tcp.accept(acceptor) # Start a process to proxy connections from clients {:ok, pid} = GenServer.start_link(InternalWorker, socket: Controlling_process (sock, PID) send(self(), :accept) {:noreply, State} end end defModule server. InternalWorker do@moduledoc """ "" use GenServer require Logger def start_link(opts) do {name, opts} = Keyword.pop(opts, :name, __MODULE__) GenServer.start_link(__MODULE__, opts, name: name) end def init(socket: socket) do :inet.setopts(socket, active: true) {:ok, %{socket: Socket}} end # received <<9, Def handLE_info ({: TCP, socket, <<0x09::8, 0x01::8, IP ::32>> = data}, state) do Logger.info("internal recv => #{inspect(data)}") IPSocketStore.add_socket(<<ip::32>>, Self ()) # handshake :gen_tcp. Send (socket, <<0x09, 0x02>>) Map.put(state, :ip, <<ip::32>>)} end endCopy the code

At this point, the part of establishing the connection handshake is complete.

The server listens to external services

In addition to listening for client connections, the server also needs to listen for external mapped ports. For example, our configuration file format is as follows:

NAT: -name: "server0" from: localhost:8080 to: 192.168.10.101:80 - name: "server1" from: localhost:8081 to: 192.168.10.101:81Copy the code

We need to listen on port 8080 and port 8081. We can dynamically create multiple listener processes according to the configuration file, the specific code is as follows:

ExternalListener do@moduledoc """ "require Logger use GenServer alias Server.{ExternalWorker, SocketStore, Utils} def start_link(opts) do {name, opts} = Keyword.pop(opts, :name, __MODULE__) GenServer.start_link(__MODULE__, opts, name: Name) end@doc """ example %{"from"=> "localhost:8080" "to"=> "192.168.10.101:80"} "" def init(NAT: NAT) do [_, port_str] = NAT | > Map. Get (" from ") | > String. The split (" : ") port = String. To_integer (port_str) # listening {: ok, acceptor} = :gen_tcp.listen(port, [:binary, active: false, reuseaddr: true]) send(self(), :accept) Logger.info("Accepting connection on port #{port}..." ) {:ok, %{acceptor: acceptor, nat: nat, port: port}} end def handle_info(:accept, %{acceptor: acceptor, port: Port} = state) do # create connection {:ok, sock} = :gen_tcp.accept(acceptor) Logger.info("new connection established from port #{port}") sock_key = Generete_socket_key () # create a worker to handle external data {:ok, pid} = genserver.start_link (ExternalWorker, socket: sock, NAT: state.nat, key: Socketstore. add_socket(sock_key) :gen_tcp.controlling_process(sock, PID) # register with key => socketStore. add_socket(sock_key, pid) pid) send(self(), :accept) {:noreply, state} end endCopy the code

The Worker handling external traffic needs to notify the client immediately after it is created. Details are as follows:

ExternalWorker do@moduledoc """ "" use GenServer require Logger alias Server.{InternalWorker, SocketStore, IPSocketStore, Typespec} def start_link(opts) do {name, opts} = Keyword.pop(opts, :name, __MODULE__) GenServer.start_link(__MODULE__, opts, name: name) end def send_message(pid, message), do: GenServer.cast(pid, {:message, message}) @spec init(socket: Typespec.socket(), nat: map(), key: Typespec.sock_key()) :: {:ok, pid()} def init(socket: socket, nat: nat, key: Key) do # parse config file, To obtain the corresponding network port number and address/client_ip_raw, client_port = NAT | > Map. Get (" to ") | > String. The split (" : ") {: ok, {ip0 ip1, ip2, Ip3}} = client_ip_raw | > to_charlist () | > : inet. Parse_address (#) is set to the first passive mode, do not receive the packet: inet. Setopts (socket, active: false) send(self(), :tcp_connection_req) {:ok, %{ socket: socket, key: key, client_ip: << IP0, ip1, ip2, ip3>>, client_port: string. to_INTEGER (client_port), status: 0, # :queue.new()}} end def handle_info(:tcp_connection_req, state) do logger. info("send TCP connecntion request") Send_msg (state. client_IP, <<0x09, 0x03, state.key::16, state.client_port::16>>) Start receiving traffic :inet.setopts(state.socket, active: {:noreply, map. put(state, :status, 1)} end def handle_info({: TCP, _, data}) State) do logger. info("external recv => #{inspect(data)}") new_state = case state.status do 2 -> # # already set :ok = send_msg(state.client_ip, <<state.key::16>> <> data) state _ -> # not set, If the state is still in the hand, or not handshake, the external traffic should be queued, Put (state, :buffer, :queue. In (data, state.buffer)) end {:noreply, new_state} end... endCopy the code

When the server sends a notification to establish a TCP connection, the client responds with the following code according to the protocol:

Def handle_info({: TCP, socket, <<0x09, 0x03, key::16, client_port::16>>}, state) do Logger.debug("selector recv tcp connection request") create_local_conn(key, Gen_tcp. send(socket, <<0x09, 0x03, key::16>>Copy the code

After receiving the reply receipt from the client, the server triggers subsequent status changes. On one hand, the server changes the status to handshake completed, and on the other hand, the server sends cached traffic to the client.

def handle_info(:tcp_connection_set, State) do Logger.info(" Recv TCP Connecntion Finished ") # send all buffer to client Flush_buffer (state.buffer, state.key, state.client_ip) # flush_buffer(state.buffer, state.key, state.client_ip) Cache to empty new_state = the state | > Map. The put (: status, 2) | > Map. The put (buffer, : the queue. The new ()) {: noreply, new_state} the endCopy the code

Flush_buffer {flush_buffer}} flush_buffer {flush_buffer}}

defp flush_buffer(buffer, key, ip) do
  buffer
  |> :queue.out()
  |> case do
	{{:value, msg}, buf} ->
	  send_msg(ip, <<key::16>> <> msg)
	  flush_buffer(buf, key, ip)

	{:empty, _} ->
	  nil
  end
end
Copy the code

Mapping relationship Management

Both clients and servers need to manage key-to-socket or IP-to-socket mappings. This is actually a global dict. In Elixir, Registry or Agent can manage this data.

defmodule Server.IPSocketStore do use Agent alias Server.Typespec def start_link(_opts) do Agent.start_link(fn -> %{} End, name: __MODULE__) end # Randomly select a socket @spec get_socket(Typespec. IP ()) :: Typespec.socket() def get_socket(ip) do __MODULE__ |> Agent.get(& &1) |> Map.get(ip) |> (fn nil -> nil socks -> Enum.random(socks) end).() end @spec add_socket(Typespec.ip(), Typespec.socket()) :: :ok def add_socket(ip, pid), do: Agent.update(__MODULE__, fn x -> Map.update(x, ip, [pid], &[pid | &1]) end) @spec rm_socket(Typespec.ip()) :: :ok def rm_socket(ip), do: Agent.update(__MODULE__, fn x -> Map.delete(x, ip) end) endCopy the code

Communication stage

Through the preparation of the previous stages, the communication stage is simple. The server only needs to know the IP address and port of the Intranet and then throw it to the client:

defp send_msg(ip, Case ipsocketStore. get_socket(IP) do nil -> logger. warn("no socket avaiable") {:error, "No socket avaiable"} pid -> internalworker.send_message (pid, MSG) #Copy the code

conclusion

We use Elixir language to build a multi-port to multi-port, and multiplexing Intranet penetration tool. In this project, we designed a simple protocol ourselves to perform the replay of TCP events on the client side and multiplex the traffic of a single escape link to multiple pairs of peers. In the concrete implementation, we use some Erlang socket programming skills, using beam virtual machine process to proxy socket transceiver action, using Agent to manage socket mapping. Finally make the whole software context clear, simple structure. Of course, this penetrating software still has many shortcomings, such as:

  1. Exception handling is not considered in the protocol;
  2. The server does not support dynamic configuration.
  3. Using: gen_TCP proxy socket does not do traffic limiting;

Of course, as a toy project, can not be perfect, used as a CS programming exercise is very suitable.