Legacy Development and Integration Guide

From RidgeRun Developer Connection
Revision as of 10:07, 18 March 2010 by Mrodriguez (Talk | contribs)

Jump to: navigation, search

Linux Application Development

This section targets software developers with a little background on Linux software development, presenting some basic concepts about Linux applications that may be unfamiliar to developers with experience in real time operating systems (RTOS).

Linux application development is more like standard PC application development and less like RTOS focused development. The main differences is the clear separation between the applications and the operating system and also that applications are stored on a file system.

This section doesn't replace a good book on Linux/Unix software development, but introduces the minimal concepts that will be useful when porting applications.

Kernel and User Space

Linux is a multi-tasking operating system with protection between executing processes.Therefore applications run in their own address-space which is provided by the kernel using the hardware memory management unit (MMU).

Since applications are separate from the kernel, they live on some file system, therefore Linux needs a least one file system holding the initial applications: the root file system.

The File System

Linux file system is based on Unix file system concepts, and therefore it is composed of a tree of files that are located on some physical storage. The physical storage should provide the root of the tree (the root file system) and other devices can be “mounted” at directories in the file system tree.

Following the Unix mindset, most of the computer universe is represented like a file: files are data containers for more than binary executable files or raw data stored on memory devices, files are also used to represent device drivers, interprocess pipes, or system status.

Unix file system layout is also important, as there are standard locations for configuration files, shared libraries, binaries, shared data, etc.

Pseudo file systems

Linux has several pseudo file systems that when mounted doesn't reflect actual stored files on some memory device, but file handlers that can provide information about the system, a process, a driver, etc. This pseudo file systems generate the file entries and contents dynamically, based on the system status.

The two more common pseudo-file systems are procfs, and sysfs. More information on those can be found on the next section.

Linux File System Layout

Understanding the Linux file system layout is important for the porting process. The following table explains what's the propose for most common locations and give you an idea or where to store the contents of your application.

Location Propose
/bin Intended for executable binaries required for the basic system.
/sbin Intended for executable binaries required for the basic system that should be available only to the super (root) user.
/lib Shared libraries for the basic system.
/etc Text configuration files of the basic system.
/tmp Temporary files: on most embedded systems this directory contents are lost between power cycles.
/var Intended to store data from system programs: web server pages, log files, mail server data, etc.
/proc This is the typical mount point of Linux's pseudo file system procfs, which creates a set of files that contain different aspects of the running system, processes, drivers, etc.
/sys This is the typical mount point of Linux's pseudo file system sysfs, which provides information on, and control of, devices drivers and kernel objects.
/opt Intended for applications that don't form part of the standard system provided by the vendor. These are usually third-party applications. Most of the applications at this location don't conform to the bin/lib separation of the Unix file systems.
/usr/bin Intended for executable binaries that are not required for the basic system.
/usr/sbin Intended for executable binaries that are not required for the basic system that should be available only to the super user.
/usr/lib Shared libraries not required for the basic system.
Embedded Linux File System

The standard Linux file system is designed for use by more than one person at a time. This is rarely the case on an embedded device, so the difference between /bin, /sbin, /usr/bin, /usr/sbin,/opt may seem less useful than for standard desktop computing proposes.

However one of the important uses of this separation is that it simplifies the final layout of the file systems used by the embedded target. RidgeRun applies the following rules to decide where to locate the applications:

  • /bin, /sbin: minimal binaries to boot the system
  • /usr/bin, /usr/sbin: standard Linux applications that are not required to boot the system
  • /opt: customer applications

This layout permits different mount points to be used for the /usr and /opt directories which allows:

  • Creating smaller minimal root file system to speed up booting (depends on the file system technology used), or that using a read only file system.
  • Simplifying software upgrades
  • Security analysis is simplified if suspect activities are limited to a small part of the file system.
Embedded target file system

RidgeRun SDK creates the target file system on the Linux host machine, and processes it to create the proper target file system image to be downloaded (if required) to the embedded system. The file system of the target system is found under the base (referred to as $DEVDIR) of your SDK: $(DEVDIR)/fs/fs.

This directory only exist once you have built the SDK for the first time, and you can copy files to this location to include them in the target file system image.

Since the file system of the target should have a minimal footprint, the SDK only puts the strictly necessary parts of the applications into this directory, leaving out some run-time unnecessary parts of many programs like include files, manual and info pages, unused localizations, etc. However some of this files (like the includes) may be required during the compilation process for other programs. For this propose the SDK provides the directory $DEVDIR/fs/fsdev, which is the typical installation point of most applications (including all the unnecessary run-time files), and later the scripts of the application copy the minimal parts from $(DEVDIR)/fs/fsdev towards $(DEVDIR)/fs/fs . More information on this procedure will be explained later on this document.

SDK build system

Building software for embedded systems is a complex task as it typically involves cross-compiling. Cross compiling presents new challenges to already established procedures for building software. The RidgeRun SDK uses a build system that simplifies the complexities of cross compiling.

The SDK build system was designed by embedded Linux programmers for embedded Linux programmers, therefore it not only simplifies the process of building embedded Linux applications, but the process of integrating, using or developing open source tools or packages with the SDK.

Introduction to RidgeRun SDK build system

Embedded Linux software development differs in various ways from the two environments it mixes: “embedded” and “Linux”.

Embedded Linux differs from most embedded environments using an RTOS in that Linux uses a file system outside the kernel, and advanced software tools like dynamic loaded libraries, multiple applications, dynamic modules, powerful networking tools and high modularity are supported.

An often overlook but crucial difference between Linux and other RTOS systems has to do with the dynamics of building and integrating open source software. Open source software is build by different communities around the world, and a typical embedded Linux system will include pieces from hundreds of developers and several different open source projects. This factor impacts various processes: system integration, application building, bug reporting, etc.RidgeRun support services minimizes this complexity by providing a single point to get access to embedded Linux support experts, but also by accounting this factor on the design of the build system.

Embedded Linux also differs from standard Linux distributions in its requirements for custom tailored kernel, drivers, and file system. Embedded limitations on footprint, power consumption and processing power create a big gap between embedded and desktop Linux systems.

All the previously mentioned details need to be taken into account by the development procedures for building and packaging the software for embedded Linux. We found three different approaches from SDK providers to build the system (with RidgeRun using the last option):

Binary software distributions: some embedded Linux providers ship the file system applications pre-compiled for the target hardware. This approach provides the advantage that the overhead and work to recompile the basic system is avoided, however also presents several disadvantages:

  • Minimal footprint is not achieved as the system is often compiled with broader-audience by including generic features.
  • The packages may have been compiled with or without features required by the customer, therefore requiring the re-compilation of certain packages.
  • Incompatibility of binary code with other software provided by third parties (For example if the object files uses different ABIs).
  • In some cases getting the proper code that matches the binary files may be challenging for debugging or tracing proposes.
  • The binary-oriented design usually lacks proper support to simplify building code from source.

Source software distributions by emulation: some embedded Linux providers use a source building based system that is compiled by running a virtual machine emulating the target hardware and using the compiler natively on the emulated target. This approach has valuable advantages:

  • Avoids the complexities of cross compiling: open source tools like autotools are in better position to properly evaluate the environment, and several other tricks required for cross compiling are not required.
  • Allows building optimized code for the target hardware.
  • Allows creating optimized footprint and features on the compiled software.
  • Permits testing on the emulated environment.

The disadvantages of this system are:

  • Overhead on setup, maintenance and build time due the simulated environment. On big systems the compilation delays my be noticeable.
  • Development files that are not required will be included on the target file system.
  • Building and maintaining an accurate emulation environment is complex and time consuming.

Source software distributions by cross compilation: consists on building the applications from source for the target hardware from the host target. The advantages are:

   ● Allows building optimized code for the target hardware.
   ● Allows creating optimized footprint and features on the compiled software.
   ● Compilation and build time are typically fast as the host machine is usually more
      powerful than the target.

The disadvantages of this system are:

   ● Requires special procedures and manual tweaking when cross-compiling complex
      applications; for example open source software using autotools requires special steps,
      since many run time detections can't be performed while cross compiling.
   ● Dependency handling between open source packages require special tools.

RidgeRun SDK build system is a source distribution built using cross compilation, and has