Android provides a user level interface for VPN services with which programmer only need to focus on the interaction with remote server.
Other actions, such as virtual interface creation, address and route configuration are done by OS.
From the APP side, there are two components for a VPN connection, the client and the service.
1. Make a VPN connection from Client
For the client, an intent for the VPN service must be required by the call of VpnService.prepare(). The intent can make sure that there is only one active VPN connection. If there is already a prepared VPN connection (e.g. stop and then start the same VPN connection manually), null will be returned. For which case, we should call the onActivityResult function manually.
Below is the code used to start a VPN service on Button’s onClick callback.
public void onClick(View v) {
Intent intent = VpnService.prepare(getApplicationContext());
if (intent != null) {
startActivityForResult(intent, 0);
} else {
onActivityResult(0, RESULT_OK, null);
}
}
After the intent finished or null returned, VPN service can be started by the calling of startService, with a intent of a VPN service as the parameter.
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
if (resultCode == RESULT_OK) {
Intent intent = new Intent(this, MyVpnService.class);
startService(intent);
}
}
2. Implement a VPN Service
There are two function must be implemented when extending the Service class, the onStartCommand and the onDestroy. Usually a Thread can be used to provide the service in background, which can be start or interrupted/destroyed in above functions.
Android creates a TUN interface for VPN service, and provides API used by APP to interact with the TUN. We should do the following to implement a VPN service:
- Use a Builder to obtain an interface (FileDescription) for the TUN. The ip address, dns, and route table can be configured via the Builder. Please make sure the addRoute is properly called, as route table determinants which packet will be routed to the TUN.
- Get a input and output stream for the interface. Input stream and output stream are used to read and write packets in a manner of stream.
- Make a connection to the VPN server. Programmer can make any kinds of logic to handle the packet, usually use a connection to tunnel packet to the remote VPN server.
- Protect the tunnel socket from the VPN service. In order to avoid the deal loop of packet (tunnel packet be routed back to the TUN other than remote server), the connection must be protected (keep untouched) from the VPN service.
- Loop with the packet.
The inputstream (in), outputstream (out), tunnel connection (tunnel) and remote server work as below:
Network Activity -> TUN -> in -> tunnel -> Remote Server Network Activity <- TUN <- out <- tunnel <- Remote Server
Below code shows these steps.
public class MyVpnService extends VpnService {
private Thread mThread;
private ParcelFileDescriptor mInterface;
//a. Configure a builder for the interface.
Builder builder = new Builder();
// Services interface
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
// Start a new session by creating a new thread.
mThread = new Thread(new Runnable() {
@Override
public void run() {
try {
//a. Configure the TUN and get the interface.
mInterface = builder.setSession("MyVPNService")
.addAddress("192.168.0.1", 24)
.addDnsServer("8.8.8.8")
.addRoute("0.0.0.0", 0).establish();
//b. Packets to be sent are queued in this input stream.
FileInputStream in = new FileInputStream(
mInterface.getFileDescriptor());
//b. Packets received need to be written to this output stream.
FileOutputStream out = new FileOutputStream(
mInterface.getFileDescriptor());
//c. The UDP channel can be used to pass/get ip package to/from server
DatagramChannel tunnel = DatagramChannel.open();
// Connect to the server, localhost is used for demonstration only.
tunnel.connect(new InetSocketAddress("127.0.0.1", 8087));
//d. Protect this socket, so package send by it will not be feedback to the vpn service.
protect(tunnel.socket());
//e. Use a loop to pass packets.
while (true) {
//get packet with in
//put packet to tunnel
//get packet form tunnel
//return packet with out
//sleep is a must
Thread.sleep(100);
}
} catch (Exception e) {
// Catch any exception
e.printStackTrace();
} finally {
try {
if (mInterface != null) {
mInterface.close();
mInterface = null;
}
} catch (Exception e) {
}
}
}
}, "MyVpnRunnable");
//start the service
mThread.start();
return START_STICKY;
}
@Override
public void onDestroy() {
// TODO Auto-generated method stub
if (mThread != null) {
mThread.interrupt();
}
super.onDestroy();
}
3. Permission
Service and Internet permission should be declared.
<uses-permission android:name="android.permission.INTERNET"/>
<application>
<service
android:name="com.example.vpnsrv.MyVpnService"
android:permission="android.permission.BIND_VPN_SERVICE" >
<intent-filter>
<action android:name="android.net.VpnService" />
</intent-filter>
</service>
</application>
4. A Packet bypass Implementation
Using the service, we can build many kinds of interesting Apps, such as Proxy, Package Filter, Bandwidth Saver. There is an example (ToyVpn) in Android Samples for SDK (file path: sdk\samples\android-17\ToyVpn ), which can also be found in google source code here.
Here we are going to implement a packet bypass function, in which without the remote server, we bypass/filter packets that go through the VPN service.
The TUN is working on OSI layer 3, so what be read and write here is the IP packet.
From APP level, Android/Java do not support raw socket, so we can not directly bypass these IP packet into network interface.
But we can protect socket from VPN service, so its possible to use a protected socket to send/get the packet. TCP/UDP socket is working on OSI layer 4, so we need do a layer translate here:
- Get IP packet from TUN. Same as all VPN service does.
- Extract layer 4 information. Protocol type (e.g. TCP/UDP) and its payload is a must. As there is a handshake procedure in TCP, before getting actually payload data from it, we need to write back handshake packet first.
- Choose corresponding socket to send out the payload. As this step is working on layer 4, so we need to save the socket and try to get return data later. If there is any return data, we need to pass these packet to TUN.
- Get packet from socket, and build a layer 3 packet. First, we need to build a valid layer 4 packet. UDP is a bit easier as the 4 byte UDP header only contains source address, source port, destination address, destination port. TCP is more complex as it’s a state connection, the sequence number and acknowledge number should be properly set. Then, use the layer 4 packet as payload, we need to build a valid layer 3 packet.
- Write IP packet back to TUN. Same as all VPN service does.
5. Suggestions
Building these IP packet is not a easy thing, the suggestion is to use a packeting filter/capture tool like Wireshark to check it first.
ByteBuffer is a good choice for packet build and value retrieve, but keep in mind that Java treat short/long as signed values, do convert it to unsigned first.
Take UDP packet first. After the TUN is connected, usually OS will send out some DNS queries, which is on UDP port 53. The nslookup command is a good test for it, if something goes wrong, a timeout error will be reported.
My name is Ramesh Natarajan. I will be posting instruction guides, how-to, troubleshooting tips and tricks on Linux, database, hardware, security and web. My focus is to write articles that will either teach you or help you resolve a problem. Read more about
Comments on this entry are closed.
Thanks a lot for this article.
Just one thing, do you have a working implementation of the packet bypass function? If yes, could you post it as example please.
Sam
This tutorial is real helpful, thanks a lot !!
But can i set user/password to connect or use a .ovpn config file to connect ? when i finish step 3, my app shows that there’s a connection to VPN, but in fact, this connection doesn’t work, i can’t realize
the step 4, is there any help ? Thanks!!
Hi,i’m having a little trouble with the following points. Could you please help me out?
— Does the TUN interface created by the VPN Service act like a tunnel, or like a capturing net that pushes packets into the actual tunnel?
— The address that is configured for the interface using the builder (builder.setSession.addAddress()) — shouldn’t it be the Local Loopback address if the VPN Service is implemented inside the app itself?
And thanks so much for posting this article!
@cougar,
What about the server/remote side? Yes, If you have go to step 3, from APP side, there is a connection. But whether it works or not depending on if you properly send packet to the remote server. Maybe you can go through the ToyVpn example first.
@Sauvik,
Tunnel is a must for VPN service. What we are implementing here is the tunnel function, the tunnel interface is done by OS. From App’s side, every packet goes to the TUN. From VPN service’s side, the packet will be send to remote server, which meet the definition of tunneling.
The address is assign to the TUN interface, it should be a local address, but not be a local loopback address. The server/remote also use the address to provide proper networking service. A local loopback address will mess up many things.
Terence
You say ‘TCP is more complex as it’s a state connection, the sequence number and acknowledge number should be properly set. Then, use the layer 4 packet as payload, we need to build a valid layer 3 packet.’
So if you are writing a bypass or a filter or the like, it seems like you have to keep track of the current sequence number and ack number expected by the client running on the client side that is talking to your bypass service, and build responses appropriately. Fortunately, you shouldn’t have losses on this link, but you still need to do a lot of state management. This looks like a very complex service to write. It woudl be much easier if one could use raw sockets !
If you want to do packet filtering or bypass, it seems like you have to basically do all TCP packet management yourself, i.e. keeping track of sequences
So I forgot to mention what I was asking — which is basically ” Is my assumption correct, or is there some way to avoid implementing a mini user-level TCP stack? if you want to do a bypass ?”
@Mudra
You are right. Using a raw socket is the right way. But it’s not possible for non rooted device. Besides, there are already some user level tcp/ip stack projects, it is possible to port one here, especially for these that used in transparent proxy.
Great article @TerenceSun
I have a similar problem as @Cougar though. My VPN Server requires me to pass a username and password to it in order to establish a connection. Any idea on how I would do that?
Thanks again
Do we have similar VPN service API in IOS? I need to implement VPN service in IOS app.
Can we somehow bypass remote server? I mean, not sending data to the tunnel but the real destination directly?
you write: while (true) {
//get packet with in
//put packet to tunnel
//get packet form tunnel
//return packet with out
//sleep is a must
Thread.sleep(100);
}
but how to get packet with in? as we don’t know the size of the packet in advance, when we read some bytes from in (inputstream), these bytes might contain more than one IP packet or might just be part of an IP packet but not the whole IP packet.
from the bytes read from in, how can we reconstruct the packets?
thanks a lot in advance if you can shed some light on this issue….
@omy
If with root access, we can write packet directly into any interface as we want.
But, with the VPN api, we can only write into the tunnel.
Please note that, the in as an instance of FileInputStream is constructed from a VPN interface. If we set buffer to the maximum size of IP packet, say 65535 byte, the integrity of packet is guaranteed. The return value of in.read records the actual size of the packet.
@Terence: thanks a lot for the reply.
I’m ok with what you replied: ‘If with root access, we can write packet directly into any interface as we want.
But, with the VPN api, we can only write into the tunnel.’
But if we set buffer to the maximum size of IP packet, say 65535 bytes, i’m ok with the fact that the integrity of the packet is guaranteed but what if there are more than one packets in these 65535 packets?
After reading from “in” stream, of course, we can write to the tunnel the bytes read. There is no problem…that’s done in ToyVpn example. But how to reconstruct/rebuild IP packets on the fly from these bytes received? I hope you got my question?
Sorry…a mistake in the previous comment: “these 65535 packets” —-> “these 65535 bytes”
And just to elaborate a bit my previous comment: when we try to read 65535 bytes, let’s say that the length of the bytes actually read is 60000. Now the challenge is to know if it’s one IP packet or there are N ip packets in these 60000 bytes or there are N + a part of N+1th ip packet?
@Terence: I might still not be clear to you…so I explain you my problem once again…
The thing is: Writing the bytes read from “in” to “tunnel” is ok. The problem is I want to play with the bytes read from “in” and convert these bytes to meaningful ip packets…and then, look at different layers (for example, TCP headers)…
Is it clearer?
Terrence & Omy
As a follow-up you article and comment string dating back to jan. ’14, i am wondering if either of you had made any progress in actually solving the noted bypass problem with ode that actually runs on Android or if anyone else had?
We have a similar need, ergo the question. If for some reason either of you has made some progress but have not completed it, we have some resources at our end that would be interested in contributing to driving the solution forward.
Tx.for the great article and directional thinking towards the solution and for any additional help/insights you maybe able to provide
Best,
d.
Hi,
We should be able to read packet from input stream and rebuild a packet and write to output stream?
Is there any example provided?
-Thanks.
we are interested in decoding ip packet and have application data read from it (i.e. HTTP/UDP) and have some gating logic applied.
any pointers to the same is appreciated
Hi, @omy
I just have the same question as yours, just wonder how is your progress?
Thanks for you great insight!
HENRY
Hi , Terrence，I’m also interested in you project . Can you send me an example . If you could send an example to my email , thanks a lot . . .
Amazing Article ,
I am looking forward with IPSec tunnelling !
Did you tried to implement secure VPN ?
Yes ashokkumar you can use similar vpn service api in IOS, however for further more information you should check review from vpnranks.com
Hi @TerrenceSun, great article. I have a question about “Besides, there are already some user level tcp/ip stack projects, it is possible to port one here, especially for these that used in transparent proxy.” I tried to find one, but I couldn’t. Basicaly, I want to use VpnService to intercept all requests and without any remote server use Google Data Compression Proxy. As I understand, I need a user level tcp/ip stack. Could you point me some projects or articles about them?
I have a Proof-Of-Concept implementation working as part of my bachelor thesis. I am searching for Co-Developers in the near future to provide some API like (And Open Source) Capture Service.
Packet fragmentation is yet currently missing, and the code is kind of overheady due to the many changes. If anyone is interested: I’m following this comments.
Cheers,
goethe
This is a great article! But it seems like we all are having trouble in completing the packet bypass functions in the while loop.
@goethe I am also working on a Master’s Thesis and what is done in this article is one of the steps of my project. I would like to exchange information and/or I would appreciate if you could help me with the packet fragmentation.
Cheers,
mete
hi
i have a question,
i want set username and password for vpn connection
how to do it?
please help me
thankyou
Any idea how to configure a Proxy into this solution?
Hi I want to use ikev2 with certificate authentication in Android. Can you help me that how can I load certificate and how can I configure VPN profile programmatically in Android. What steps I need to do as I am new to Android. Thanks in Advance.
hi
how can i connect to a pptp server with a username and a password?
Hi ,you had said:
Using the service, we can build many kinds of interesting Apps, such as Proxy, Package Filter, Bandwidth Saver.
These function are interesting and have you finish some of them.
I have established a local vpn in my android device with
Builder builder = new Builder(); builder.setMtu(1500) .setSession("abcd") .addAddress("10.0.0.2",32) .addDnsServer("8.8.8.8") .addRoute("0.0.0.0",0) .setBlocking(false); return builder.establish();But after that non f the apps in the device can access internet.
Do I need to explicitly pass the packats to and from the tunnel ?