Summer of Code Project (InstantMirror)

Finally, i am writing about my this year google summer of code project InstantMirror.

From wiki of InstantMirror

InstantMirror is meant to solve repository replication problem in efficient way.Various modern techniques are utilized and combined like Squid-like reverse proxy cache and cache expiry algorithm, rsync-like local directory trees, Torrent style efficient swarming many-to-many data replication. This project if implemented would dramatically improve the efficiency and timeliness of repository replication. This could be very useful for mirror networks like kernel.org, Fedora, Debian, CPAN, etc.

This means, if I am able to complete the project this can be helpful in various organizations for replication.

For more technical details about the project visit Wiki or hangout on #instantmirror irc channel of freenode or join Mailing List for development details.

ProxyChains Library

A less than a week and I will be away from the university and hence away from proxy server of computer centre, but still I am looking for solutions to bypass proxy and to tunnel ports blocked by proxy.

I found an amazing library named ProxyChains which can establish chain of proxy for your communication and bypass all the communication and tunnel the required port to avoid filterning. I already published proxy tunneling throug transconnect library, so what’s difference between them. Well the difference is that transconnect does not support SOCKS proxy (which is the main proxy used to tunnel through ssh) whereas ProxyChains allow it. Also there are several other features in ProxyChain but the only reason for me to use it is  SOCKS support.

So the beginner’s question, How to bypass proxy and tunnel proxy?

Step 1. Get an ssh shell account on free shell provider. List of free shell providers are http://www.red-pill.eu/freeunix.shtml andhttp://www.bylur.net/free/.

Step 2: Check whether free shell provider allows port forwarding. Also check where http ports are allowed or any port is allowed. To check first ssh login in shell using

ssh username@host -D 1080 -v

and then use the socks proxy in firefox to localhost and port 1080 and try to browse the web. If you are successful, atleast http ports are forwarded. Now try to connect to some random port in the firefox like something:2354 and see what type of message appears on the verbose ssh shell. If it tries to connect, then you have found the right shell and proceed further otherwise try to find another shell. Also keep note down which shell give http port access. You may use it to browse net when proxy is down or something.

Step3: Install ProxyChains (http://proxychains.sourceforge.net/) and set SOCKS proxy in the proxychains.conf file usualled located in /etc and start using it.For more information read Readme file or http://proxychains.sourceforge.net/howto.html.

You can also use this SOCKS proxy in every application which support SOCKS proxy like firefox etc. and use ProxyChains in every other case like which does not have SOCKS support or no proxy support at all.

In my institute (i think usually in all institutes), ssh port is automatically forwarded and hence does not require any proxy. Hence this is a very reliable method.and even works when proxy is down due to power down or restart.

In case you feel some difficulty and any of steps is not clear, please leave comment. i will try to explain and elaborate.

How to set proxy in terminal

Setting proxy in terminal is very straight forward. Use export command to set it.

export http_proxy=http://proxyip:proxyport

Replace proxyip with the ip of the proxy and proxyport with the port of proxy.
In case your proxy requires authentication, use

export http_proxy=http://username:password@proxyip:proxyport

For example, the command becomes (in my university)

export http_proxy=http://282421:pass@10.1.1.19:80

You need to enter this command in each terminal you open or you can enter this in ~/.bashrc.
After setting proxy in the terminal, you may now use wget or apt-get/yum without getting error Connection refused.

You may be wondering, why I am posting only on proxy. The only reason is i got struck on proxy settings on my first installation of linux and removed it. It took a long time when i started using linux again. Also, i am behind a proxy server in my college and aim to provide simple articles for beginners.

My Last GSOC (Adding proxy support in Sip Communicator)

Last year (2008), i got selected as google summer of code student for Sip Communicator for the project Adding Http/Socks proxy for Sip Communicator. That was my first contribution to the open-source world.

I completed the project with adding proxy support to several protocols in sip communicator like jabber, rss feed, ssh, aim, icq, irc etc. However, i could not add proxy support in yahoo and msn. Both of these requires high refracting of their respective library.

During the course of project, i modified java jabber library smack to add proxy support and sent the patch to library which accepeted by patch to integrate it into smack trunk. I also sent the patch to pircbot (irc java library) but didn’t got any reply from it.

Hence during the course of summer of code project, I did contribute to smack, pircbot, sip communicator. However Sip Communiator has not integrated proxy support in the trunk yet and has planned to release this feature in 2.0. The code integration will start once the branch for 1.0 release is segregated from the main trunk. I need to merge the code into current trunk as the trunk has been changed a lot since my last commit to proxy branch.

If you want to try the proxy branch you may checkout the code from sip communicator using the command

svn co https://sip-communicator.dev.java.net/svn/sip-communicator/branches/proxy

However this branch does not contain current trunk and hence you will have to face the bugs which has been resolved now or nice gui which has been setup after my last commit. I am trying to integrate the proxy code into current trunk in branch named proxy2. You may download the code from it but as per now, it has only global proxy config plugin, and proxy support for one protocol. I will try to complete it as soon I have some time.

Linux applications behind proxy

There are tons of applications in linux which does not have proxy support and using them behind proxy is not an easy task. I have been in search for Proxifer alternative for linux, but couldn’t find it. I tried some softwares for it but they didn’t perform as required.

I was trying to checkout the source code of koffice (for gsoc purposes) and found the library which made it possible. The library is transconnect which is hosted on sourceforge.net.  The steps to use this library are:

1. Download the latest source code from the sf.net website. (You need to manually build as binaries are not distributed).
2. Build the binaries by using make and install the lib. using make install.
3. Edit the ~/.tconn/tconn.conf file to include the proxy host,port, username and password to be used.
4. If you need all the traffice to be routed through the proxy server, type export LD_PRELOAD=~/.tconn/tconn.conf in the terminal and start the application by the terminal.

If you don’t want to use the proxy server you may type unset LD_PRELOAD and now all the traffic will be sent as per the application.

One point which is to be noted is that you need to set LD_PRELOAD manually for every terminal you use or you may include this command in the .bashrc file.

Also if you need the proxy support in some graphically application, you need to start the application from the terminal only.

Transparent network over HTTP (transconnect) is a great application and more information can be seen in the sf.net page of Transconnect. Transconnect is licenced under GPL licence.

Running windows application behind proxy

There are several applications (of course network app.) which does not support for proxy servers. So it almost becomes impossible to use those applications. I have faced these problems for a long time before I found the solution. One of the these kind of applications is Microsoft Internet Games. It has no support for proxy settings and hence cannot be played behnd proxy.

The software which can direct all the direct connection traffic is Proxifier which is now paid. You can add several proxy chain to this software and all the traffic is redirected through the proxy chain.

Configuring Proxifier is quite simple, just add the proxy server in the proxy chain and you are done with the configuration. However you must check the proxy while adding as only those proxy which support CONNECT methods are allowed. You should also add the local ip range whose traffic should not be routed through the proxy chain. Proxifier has the functionality of adding applications/ip ranges which must be/must not be proxified.

P.S. I didn’t know proxifer is paid now, maybe because I am running a beta/old version of the software. I have found some alternatives to Proxifier like Hummingbird. I will review them and post the information.

Proxy setting in Subversion

Subversion is an important tool which needs to be configured to use proxy in case of user is behind proxy server. To add proxy support in svn edit the file ~/.subversion/servers (On Windows it is in “%APPDATA%Subversionservers”) and add the required proxy settings in the [global] group. You can also set the proxyusername and proxy password in the [global] group in case your proxy needs authentication.

After editing your servers file will look as.

[global]

http-proxy-host = proxyip

http-proxy-port = proxyport

http-proxy-username = username

http-proxy-password = proxypassword

If you are experiencing the error as:

svn: path to server file:line Section header expected

You have put the proxy configuration in wrong place, i.e. not under [global] group. Try commenting the line number mentioned and put the proxy information in [global] group to rectify the problem.

For more information see FAQ (subversion)

If you use tortiosesvn as your svn client in windows, adding proxy is one minute job (may be less). Follow these steps:Get the proxy settings from your Network Administrator or ISP

1. Right-click on the desktop or in any folder to get the context menu, select the ‘settings’ option under the TortoiseSVN menu item
2. Select the ‘Network’ section in the left pane of the window
3. Check the ‘Enable Proxy Server’ checkbox
4. Enter in the proxy server settings, and you are done.

Why this username !!!

I tried several username like atul, atulaggarwal, but these username are mostly not available on webmail or blogging sites.

Also I tried to annomize my  username and choose sylar from Hero tv serial but that username is also not available. My aim was to choose a username which gets almost everywhere available and hence finally decided to use atulagrwl as my username.

I registed this username on gmail and got the email account, same is the case with freenode (irc) and even, i got this username for the wordpress blog. So I have decided to use this username for all my future registration. I am still using another gmail account as my primary gmail account but plan to move to atul.agrwl gmail account soon.

Hello blogging world! :)

Hello World.. I thought, i will not delete this first post posted by wordpress by default but only edit it.

This is not my first blog. I tried blogging on several blogs but couldn’t update it. I thought of again trying and hence is atulagrwl.

I have thought of buying atulagrwl [dot] com domain. So i am looking for some cheap or probably free webhosting. If you know any, please contact me or leave a comment.

I will try to put my experience regading software (mostly opensource here). I will try to put tweeks (some simple settings) to make thinks work.