Yocto RR SDK Integration

From RidgeRun Developer Connection
Jump to: navigation, search

Overview

The $DEVDIR/bsp/classes/yocto.class integrates pre-built yocto packages into the RidgeRun SDK. As customers request packages, a RidgeRun engineer performs a standard yocto build creating the needed package. The files assoicated with the package are uploaded to the RidgeRun website (e.g. https://www.ridgerun.com/yocto-packages/armv7a-vfp-neon where the last directory is the target processor). In addition the md5sum files are also uploaded and the package is added to the Config.yocto file that resides in that same directory on the RidgeRun webserver.

When the user runs make config and enters the $DEVDIR/fs/yocto-packages sub-menu, the SDK logic will have prefetched the Config.yocto file from the RidgeRun webserver so a menu list of available packages can be presented to the user. During the make build build process, the yocto.class will fetch all the pre-built files assoicated with the package, save the files in the appropriate downloads directory, using the SDK's fetcher class. In addition the the developer yocto package is installed in fsdev. When the make install target is run, yocto.class will install either the release package or debug package in the target file system.


Deployed objects

  • Target fs contents
  • Target fs contents for debugging
  • Developer contents for fsdev
  • Source
  • License


Example RidgeRun copyright.xml file

<?xml version="1.0" encoding="UTF-8"?>

<component>
	<name>alsa-lib</name>
	<version>1.0.24</version>
	<releaseDate>31.01.2011</releaseDate>
	<manufacturer>ALSA</manufacturer>
	<summary>Alsa-lib is a user space library to simplify access to the kernel drivers provided by the ALSA subsystem.</summary>
	<link>http://www.alsa-project.org/</link>
	<issues>
		<link><![CDATA[https://bugtrack.alsa-project.org/alsa-bug]]></link>
	</issues>
	<license type="LGPL-2.1">
		<link>https://spdx.org/licenses/LGPL-2.1</link>
		<text>
...

		</text>
	</license>
</component>


Example Yocto bitbake recipe

DESCRIPTION = "Alsa sound library"
HOMEPAGE = "http://www.alsa-project.org"
BUGTRACKER = "https://bugtrack.alsa-project.org/alsa-bug/login_page.php"
SECTION = "libs/multimedia"
LICENSE = "LGPLv2.1 & GPLv2+"
LIC_FILES_CHKSUM = "file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34 \
                    file://src/socket.c;beginline=1;endline=26;md5=11ff89a8a7a4a690a5c78effe8159545"


References

What is Yocto Project?

The Yocto project is a workgroup (from the Linux Foundation) with the goal to provide inter-operable tools, metadata, and processes that enable the rapid, repeatable development of Linux-based embedded systems. The project aligned itself with OpenEmbedded, an existing framework with similar goals, with the result being The OpenEmbedded-Core Project.

The project has the aim and objective of attempting to simplify the process of develop customized Linux systems, supporting the ARM, MIPS, PowerPC and x86/x86_64 architectures. A key part of this is an open source build system, based around the OpenEmbedded architecture, that enables developers to create their own Linux distribution specific to their environment. This reference implementation of OpenEmbedded is called Poky and consist on the bitbake build system and meta-information files.

Yocto build system workflow

The figure 1 shows the workflow of the build system used by Yocto when creating a image for a given platform, this workflow use bitbake as the main engine. Bitbake is a highly efficient task execution system which automates the process of creating a working Linux operating system, this way using the information provided on the package recipe we can set the tasks needed for the deployment of the package in the target file system.

Figure 1. Yocto build system workflow (Source [2])

As the figure shows, the build system (bitbake) uses mainly four sources of configuration besides the source mirrors:

  • User local configuration, for settings like where to store the downloads or the resources available for the build process.
  • Software via source recipe files which specify the package location, license, checksum and specialized instructions for building.
  • A layer to define the hardware configuration including footprint size,machine architecture, user-interface definition, and board-specific configuration.
  • The distribution policy.

Once configured, bitbake knows what it needs to locate, where to get it, and what to do. The sources can com from external or local projects source code management(SCM), or source mirrors. Patches are then applied based on your customized instructions and then the system uses the Shared state to identify only altered packages (when its not a build from zero), for faster compilation. The build result can be packaged into any of the most commonly used package formats (.deb, .rpm or .ipk). Then, a sanity check is performed on each package to make sure it has been built according to configuration requirements, a directory or database of packages is created containing all compiled binary packages and a bootable linux system is generated from the package feed. This system can take several forms, from a raw disk image to a "live" image which can be booted from a usb memory stick An application developer SDK is created which in includes a cross-architecture compiler and tools, analytic tools, and an eclipse plug-in

Using Yocto

To understand how to create a Yocto working directory be sure to check references [1] and [2], there you will find a step by step guide on how to get started for the Beagleboard-xM and the iMX6 SabreLite. In this section you will find a a more specific approach for quick configuration and some of Yocto basic components.

Recipes

In Bitbake, each software package needs a recipe to specify some of its information and how it must be handled by bitbake. In a poky distribution, each layer consist of the organization of several recipes. Here is a example the meaning of the information provided by a recipe.

 #Package specific information
 DESCRIPTION = ""
 HOMEPAGE = ""
 LICENSE = ""
 #Package dependencies
 DEPENDS = ""

 SRC_URI = " \
 "
 # SRC_URI could also point to a git repository, eg:
 # SRC_URI = " git://host:port/path/to/repo.git;branch=win;protocol=ssh;user=username"
 
 # any .patch files included here will be auto applied.
 # SRC_URI = "file://omap_ctrl_readl.patch"
 
 PR = "r0"  # Package Revision, Update this whenever you change the recipe.
 
 # For tarball packages (as opposed to git / svn which include the commit in the URI)
 SRC_URI[md5sum] = ""
 SRC_URI[sha256sum] = ""
 S = "${WORKDIR}/CHANGEME-${PV}"

 #Any functions writed here will overwrite the class functions

 do_configure () {
   ./configure --prefix=${prefix}
 }
 do_compile () {
   make
 }
 do_install () {
   DESTDIR=${D} oe_runmake install
 }

 # Any inherit classes will define the functions that are going to be perform for a specific package, the class called "base.bbclass" is inherited by default

 inherit autotools

You can also find other useful variables on the Poky Reference Manual

Layer Management

When using a poky distribution (which is the distribution supported for the Yocto project) you have to specify the layers that are available for usage. This can be done in the file called bblayers.conf on your {TOPDIR}/conf/ directory. Here is a example of how to add a layer called exampled layer to the bblayers.conf file. kkk

LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  ${TOPDIR}/../meta \
  ${TOPDIR}/../example \
  ${TOPDIR}/../meta-yocto \
  ${TOPDIR}/../meta-yocto-bsp \
  "
BBLAYERS_NON_REMOVABLE ?= " \
  ${TOPDIR}/../meta \
  ${TOPDIR}/../meta-yocto \
  "

RidgeRun SDK and Yocto integration

Figure 2 shows a flow chart describing the process of integration of a package created with Yocto on the RidgeRun SDK. The .deb files created using yocto for a specific architecture are stored on a tarball and hosted in RidgeRun repository, if the user decide to use those pre-built packages (selecting them on the RR SDK menu config) the tarball is fetched, unpack and install on the target file system using the class 'yocto.class' to manage this process.

Figure 2. Flow chart of the process of integration of a Yocto package with the RidgeRun SDK

Package Deployment

The SDK handles any package dependencies for the use of a specific software package. When the user select a package, the SDK downloads a tarball which contains the source code, the license information and the target specific pre-built binaries and libraries. This package is handled by the yocto class which handles the deployment on the target file system and the development file system. The information contained in the tarball is showed in Figure 3.

Figure 3. Content a Yocto integration tarball

As a brief description of each entry;

  • Deb packages: yocto pre-built debian packages.
  • Source Code: All the source code for the prebuilt packages.
  • License: License information of the package.
  • Scripts: A directory which contains scripts to add any special consideration or change needed from a yocto-prebuilt package, this scripts will be automatically executed by the yocto integration tool. This could be used to add or modify initialization scripts on the target filesystem.

Creating tarball with source code, license and related

Since we dont want to create all the tarballs containing the source code, patches, and others needed by the open source community we use the yocto archiver class, a short description of the archiver class:

From archiver.class at yocto:

# This file is used for archiving sources, patches, and logs to a
# tarball. It also output building environment to xxx.dump.data and
# create xxx.diff.gz to record all content in ${S} to a diff file.


In order to use the class and the yocto process that already exists for creating this tarball it is just needed to specify at the local.conf file located at yocto: build/conf/local.conf

ARCHIVER_MODE ?= "original"
ARCHIVER_CLASS = "${@'archive-${ARCHIVER_MODE}-source' if ARCHIVER_MODE != 'none' else
}"
INHERIT += "${ARCHIVER_CLASS}"
SOURCE_ARCHIVE_PACKAGE_TYPE = "tar"

This will use the archiver mode "original" in order to create the tarball, other options:

  • configured
  • original
  • patched


By adding this to the config file, this will deploy files at: DEPLOY_DIR/sources (ususally: build/tmp/deploy/sources/). For example when doing this for PERL at: build/tmp/deploy/sources/arm-poky-linux-gnueabi/

.
├── GPL-1.0
│   └── perl-5.14.3-r1
│   ├── perl-5.14.3-r1.diff.gz
│   ├── perl-5.14.3-r1-series.tar.gz
│   ├── perl-5.14.3-r1.showdata.dump
│   └── perl-5.14.3.tar.gz


Idea: Use find in order to get package location and license type, example:

caguero@caguero:~/devdirs/yocto/yocto-dylan/build/tmp/deploy/sources/arm-poky-linux-gnueabi$ find . -iname "perl*"
./GPL-1.0/perl-5.14.3-r1
./GPL-1.0/perl-5.14.3-r1/perl-5.14.3-r1.diff.gz
./GPL-1.0/perl-5.14.3-r1/perl-5.14.3-r1.showdata.dump
./GPL-1.0/perl-5.14.3-r1/perl-5.14.3.tar.gz
./GPL-1.0/perl-5.14.3-r1/perl-5.14.3-r1-series.tar.gz


More information at: www.yoctoproject.org/docs/1.4.2/dev-manual/dev-manual.html#providing-the-source-code

Package selection on the SDK Config menu

If you have a Irazú branch RidgeRun SDK of a platform which have support for this feature, then the package selection menu should be available from your configuration menu; to get to the selection screen, on the root directory of your SDK type:

menu config

Figure 4 shows the option available under the "File system configuration menu" on the SDK user interface once the config option is enable (At the time of writing this article none of the platforms have this options enable by default but this can change).

Figure 4. Yocto binary support option on the SDK menu

Once enable, you can access the binary package support sub-menu. Figure 5 shows the Yocto binary support sub-menu on the SDK user interface, notice that under "available yocto packages" comment there is a list with all of the available packages, here you can select which ones you want on your target file system. If you enable the Auto-update option, every time you open the SDK configuration menu the SDK will check for new packages.

Figure 5. Yocto binary support sub-menu on the SDK menu

Using a diferent Toolchain

When creating pre-built packages with Yocto sometimes you may need to use a "third-party" toolchain, this toolchain should be distributed with it's own binary executables and libraries and since it's not part of the normal target build process you'll need to create a layer to manage the adequate usage of this external tool. Thankfully, since the Yocto-Project have a extensive support community; most of the "popular" toolchains already have a supportive layer for their use on a yocto build environment.

Here are are some of the layer available for external toolchains:

meta-sourcery

git clone https://github.com/MentorEmbedded/meta-sourcery.git -b <branch>

meta-linaro

git clone git://git.linaro.org/openembedded/meta-linaro.git -b <branch>

meta-arago

git clone git://arago-project.org/git/meta-arago.git -b <branch>

You will also need to set some variables on your local.conf file. As an example, to use linaro toolchain layer you will need to add to your local.conf

TCMODE = “external-linaro”
EXTERNAL_TOOLCHAIN = “/path/to/linaro”

Also, specifically for newer linaro toolchains you will need to add:

ELT_TARGET_SYS ?= “arm-linux-gnueabihf”

Since the latest binaries are ARM hardfp. Also be sure to add the layer to your bblayers.conf file.

Naming conventions of the integration tarball for the RR-SDK

If a client request a specific yocto package that is not already supported, it will be created to be uploaded to the web server. The name of the tarball should use the following name convention:

<Package name>-<Package version>.tar.gz2

The tarball will be uploaded to repository of the architecture of which was build for. From the user end, the only updated file will be the Config.yocto file, this file should be automatically updated (if the auto-update option is selected) by closing and opening the the config menu. The config option should use following convention:

FS_YOCTO_PKG_<PACKAGE NAME>-<VERSION>-<REVISION>

If you want to review the changes of the Config.yocto file, and it should be available from your "Available yocto packages" menu, described on the section Package selection on the SDK Config menu

Generating the contents of the tarball using Yocto

There are tools available for generating a tarball for the RidgeRun SDK, if you have interest in using a specific package, please contact us