DSLinuxSVN

Introduction

This page explains the structure of the DSLinux Subversion repository, and how to work with it as a developer.

You should be aware of everything mentioned below if you are a committer, but you may also find it interesting if you are creating patches using anonymous svn. In any case, having read this page, you will hopefully have no problem finding your way around the DSLinux source tree, including its various branches.

If you don't know about Subversion yet, this page may be a bit hard to understand. I recommend you read the freely available Subversion book available at http://svnbook.red-bean.com.

Web Interface

There is a web interface to the Subversion repository: http://svn.dslinux.org/viewvc/dslinux/

Commit Notification

If you want to get email whenever a commit is made to the Subversion repository, you can subscribe to the dslinux-commit mailing list here: http://mailman.dslinux.in-berlin.de/mailman/listinfo/dslinux-commit-dslinux.in-berlin.de

History

Back in the 2.6.9 and earlier days of DSLinux, DSLinux was based on a release tarball of uClinux, and a CVS repository at sourceforge.net was used to host files that were modified.

So to get the DSLinux sources, you had to get the tarball, unpack it, checkout the modified files from the CVS repository and copy them into the unpacked source tree created by the tarball.

This approach made it a bit complicated for people to create patches for DSLinux. Nowadays, creating patches for DSLinux is much easier.

Because of various shortcomings of CVS, DSLinux developers decided to switch to a better version control system. Git and Subversion were both evaluated as alternatives, and git was our first choice. But because there were issues converting the CVS repository to Git, we ended up using Subversion instead.

Branches in the repository

The DSLinux Subversion repository has several types of branches. Each type of branch has a specific purpose.

Vendor branches

A vendor branch is created for each vendor DSLinux uses code from. Examples of such vendors are the uClinux team, the ncurses developers, and of course the original Linux kernel hackers.

On each vendor branch, you can find unmodified versions of the source code as supplied by the vendor. This may not seem very useful at first glance. But it is indeed useful when newer versions of vendor code are imported into DSLinux. Having the old and the new unmodified original sources on a separate branch makes it easier to merge changes made in vendor code into DSLinux without destroying the modifications DSLinux developers have made. In fact, Subversion does most of the work for us.

These branches reside in the branches/vendor/directory of the repository.

The trunk

The trunk is the main branch in a repository (also referred to as the head branch or simply head). This is where active development of DSLinux takes place. Usually, checking out a copy of the DSLinux from the repository means checking out the trunk. The trunk unites all the code found in vendor branches and contains the modifications needed to make the vendor code work in DSLinux. Code that was written specifically for DSLinux (for example, device drivers for NDS hardware) can be found here.

The trunk resides in the trunk/directory of the repository.

Stable branches

Stable branches are created whenever huge changes happen on the trunk, so huge that things will break and it will take a while to get the tree in order again. This way, users can use builds made from the stable branch, and developers can continue work on the trunk undisturbed, while still being able to add small fixes and features on the stable branch.

There is only one stable branch as of today. It is the branch for the version of DSLinux based on Linux-2.6.9, and can be found in the branches/stable/dslinux_2_6_9directory. It was used while both the 2.6.14 kernel and a new uClinux userland where integrated into DSLinux. The 2.6.9 branch has now outlived its usefulness.

Overview

Directory structure

This is a simplified overview of the directory structure of the repository. Not all branches and tags are shown here. Their number is increasing over time anyway. Note that you can browse the full and current version via the webinterfaceto the DSLinux Subversion repository.

 branches/ stable/ dslinux_2_6_9/ vendor/ autologin/ bitchx/ bsdgames/ dialog/ dropbear/ frotz/ ... uclinux/ wireless_tools/ tags/ autologin/ autologin_1_0_0/ bitchx/ bitchx_1_1_final/ bsdgames/ bsdgames_2_7_13/ dialog/ dialog_1_0_20060221/ dropbear/ dropbear_0_48_1/ frotz/ frotz_2_43/ ... uclinux/ uclinux_20051014/ wireless_tools/ wireless_tools_27/ trunk/ .svnignore COPYING Makefile bin/ bulkbuild.sh config/ include/ lib/ linux-2.6.x/ toolchain/ tools/ uClibc/ user/ vendors/

A diagram

To get a better picture of how the vendor branches and the trunk interact, take a look at the following diagram.

Vertical lines represent branches of the repository.

Whenever a vendor branch is updated to a new version of the vendor code, the changes that come with the new version are merged into the trunk so that DSLinux can benefit from them. Arrows indicate such merge operations, so whenever an arrow points from one branch to another, code was merged from one branch into the other.

You can also see tags. Tags are essentially frozen copies of a state of a branch at a given time, and have same name as the vendor branch they reside on, but with a version number appended. Tags are very important for generating diffs (also called patches) between different versions of code, and thus they are very important for merge operations.

 sl linux trunk uClinux ncurses bsdgames | | | | | | | | | | | | the past | | | | | | | | | | | | sl_3_03 | | | | | | | | uClinux_20051014 | | | linux_2_6_14_hsc0 | | | | | | | | ncurses_5_4 | | | | | | | | | | | | bsdgames_2_7_13 | | | | | | | |------------>| | | | | | |<-----| | | | | |<-----------------| | | | |<-------------------------| |---------------------->| | | | | | | | | | | | | | | | | | | | | | the future | | | | | | | | | uClinux_YYYYMMDD | | | | | | | | | linux_X_Y_Z | | | | | | |<-----| | | | | | | | | | |------------>| | | | | | | | | |

Using the repository

Note that most example commands below apply only to committers, because these people are most likely to be referring to this page regularly. Committers are people who have write access to the DSLinux source tree. Most people are not committers, and they will access the repository anonymously. Anonymous access to a Subversion repository is read-only, and is often referred to as anonsvn access.

See Compiling DSLinuxfor information on how to get an anonymous checkout of the source tree.

IMPORTANT: If you are not a comitter, substitute svn+ssh://svn.dslinux.org/svn/dslinux/in all example commands below with the appropriate URL of the anonsvn mirror you are using. For example, if you are using the svn.dslinux.org anonsvn mirror, the command

 svn checkout svn+ssh://svn.dslinux.org/svn/dslinux/trunk dslinux

becomes:

 svn checkout http://svn.dslinux.org/svn/trunk dslinux

Initial checkout

Committers should put their username into the ~/.ssh/config file:

 Host svn.dslinux.org User PutYourUserNameHere

Now you can checkout a working copy by running:

 svn checkout svn+ssh://svn.dslinux.org/svn/dslinux/trunk dslinux

Importing new versions of third party software

Vendor branches are being merged into the trunk whenever a new version of vendor code is imported.

For example, assume you wanted to update bsdgames from version 2.7.13 to 2.8.0. The procedure would go like this:

Checkout a copy of the bsdgames vendor branch:

 svn checkout svn+ssh://svn.dslinux.org/svn/dslinux/branches/vendor/bsdgames bsdgames

This will create a workspace in a new bsdgames directory in the current directory. Next, make this workspace look exactly like a pristine copy of bsdgames-2.8.0. This can be done in numerous ways, like applying a patch, or copying files. Make sure to remove files that have been removed in the new version with the svn remove command, and to add files that have been added since the last version using the svn addcommand.

Now, commit bsdgames-2.8.0, and create a tag:

 cd bsdgames svn commit -m "adding bsdgames-2.8.0" svn copy svn+ssh://svn.dslinux.org/svn/branches/vendor/bsdgames svn+ssh://svn.dslinux.org/svn/tags/bsdgames/bsdgames-2.8.0

(Note: In the past when we used CVS, there was a convention to use only underscores as seperators in tag names. Since we are now using Subversion, and tags are hence just directories, we can use the exact version number scheme as made up by the upstream developers. So bsdgames-2.8.0 can stay bsdgames-2.8.0 and must not necessarily be changed to bsdgames_2_8_0.)

Now the new version is in the tree, on the bsdgames branch. Next, let's merge it into the trunk. If you do not have a trunk workspace around, create one as usual:

 svn checkout svn+ssh://svn.dslinux.org/svn/dslinux/trunk dslinux

Now, perform the merge:

 cd dslinux/user/games/bsdgames svn merge svn+ssh://svn.dslinux.org/svn/tags/bsdgames/bsdgames_2_7_13 svn+ssh://svn.dslinux.org/svn/tags/bsdgames/bsdgames-2.8.0

This will merge most changes automagically. If there are any conflicts between the DSLinux-specific changes and the changes made by the bsdgames team, proceed as described here.

Once you have bsdgames-2.8.0 working on DSLinux, commit your changes to the trunk:

 cd dslinux/user/games/bsdgames svn commit -m "bsdgames-2.8.0 working"

Before adding new third party software

Before adding new third party software, please make sure that it is not already in the tree. The uClinux branch has a wealth of software, and only small parts of it are on the trunk. See herefor an overview of userland software that is already in the tree and can be merged into the head branch easily.

Let's say you wanted to merge the editor elvis-tiny (a small vi clone) from the uClinux branch into your workspace to try if it works with DSLinux. Try this:

 cd dslinux/user svn copy svn+ssh://svn.dslinux.org/svn/dslinux/tags/uclinux/uclinux_20051014/user/elvis-tiny ./

Caution:

Do not merge anything from the branches/ directory into your working copy! USE ONLY _TAGS_ AS SOURCE OF A MERGE OPERATION, DO _NOT_ USE BRANCHES AS SOURCE!

This is important because Subversion does not track merge points automatically. If you merge from branches/vendor/uclinux/ instead of tags/uclinux/uclinux_<version>, it will be almost impossible to find out later which version of elvis-tiny you merged into the trunk, since the branch is a moving target. The state referenced by the tag is fixed, however, and will always stay the same. See also here.

Now you have elvis-tiny in your working copy, Select it from the configuration menu as usual to compile it along with the rest of DSLinux.

Once you got elvis-tiny working, you can commit it to the trunk, along with your modifications to make it work, if any. Please mention the tag you were merging from in the log message:

 cd dslinux/user/elvis-tiny svn commit -m "Merging elvis-tiny from tag uclinux_20051014 into trunk"

Adding new third party software

Let's say you wanted to add hexedit-1.0 as a new tool in DSLinux.

First, unpack the hexedit-1.0 source code somewhere, for example:

 tar -C /tmp -zxf hexedit-1.0.tar.gz

New third party software requires a new vendor branch. We create it first before doing anything else, by importing the new directory and all files within it onto a new vendor branch:

 cd /tmp/hexedit-1.0 svn import svn+ssh://svn.dslinux.org/svn/dslinux/branches/vendor/hexedit/

Next, create a directory for tags that refer to the newly created vendor branch:

 svn mkdir svn+ssh://svn.dslinux.org/svn/dslinux/tags/hexedit -m "creating directory for tags on hexedit branch"

Next, create a tag that identifies the version of hexedit you imported:

 svn copy svn+ssh://svn.dslinux.org/svn/dslinux/branches/vendor/hexedit \ svn+ssh://svn.dslinux.org/svn/dslinux/tags/hexedit/hexedit-1.0 -m "tagging hexedit 1.0"

Next, merge hexedit from the tag you've just createdinto a workspace of the trunk to make DSLinux specific changes there.

Caution:

Do not merge anything from the branches/ directory into your working copy! USE ONLY _TAGS_ AS SOURCE OF A MERGE OPERATION, DO _NOT_ USE BRANCHES AS SOURCE! See this sectionfor details.

Now, finally, the commands to do the merge:

 svn checkout svn+ssh://svn.dslinux.org/svn/dslinux/trunk dslinux cd dslinux/user svn copy svn+ssh://svn.dslinux.org/svn/dslinux/tags/hexedit/hexedit-1.0 hexedit

Make sure the program compiles correctly and runs correctly.

Give credit where credit is due! If you are committing a patch that was contributed to DSLinux by someone who does not have commit access, make sure you mention the name of that person in the commit message. A phrase like "Patch contributed by Const Contributer" is sufficient.

Now commit your changes.

 cd dslinux/user/hexedit svn commit -m "hexedit for DSLinux :D"

The vendor branch you have created will be used again when importing the next version of hexedit.