A new FreeBSD quarterly status report has been published! Among the many reports, some selected new statuses are:
- Status of Summer of Code projects
- News on Capsicum security capabilities
- ZFS, GELI and HAST news
- Userland DTrace
- CLANG replacing GCC in the base system
- New TCP congestion control algorithms
- Support for USB 3.0
Read the whole report for details!
FreeBSD Quarterly Status Report
This report covers FreeBSD-related projects between July and September 2010. It is the third of the four reports planned for 2010. During this period, we were victims of one of the biggest BSD events of the year — EuroBSDCon. We hope that the ones of you who have been able to attend it have enjoyed your stay. Another good news is that work on the new minor versions of FreeBSD, 7.4 and 8.2, is progressing well.
This report, with 55 entries, is the longest report in the whole history and shows a good condition of the FreeBSD community.
Thanks to all the reporters for the excellent work! We hope you enjoy reading it.
Please note that the deadline for submissions covering the period between October and December 2010 is January 15th, 2011.
Google Summer of Code
- Atheros AR913x SoC Support
- Binary Package Patch Infrastructure — pkg_patch
- ExtFS Status Report
- Packet Capturing Stack — ringmap
- Registration of Optional Kernel Subsystems via sysctl
- BSD# Project
- BSNMP Enhancements
- Capsicum: Practical Capabilities for UNIX
- Clang Replacing GCC in the Base System
- DAHDI/FreeBSD Project
- External Toolchain Support
- GELI Additions
- gptboot Improvements
- HAST (Highly Available Storage) Improvements
- Kernel-level Stacked Cryptographic File System — PEFS
- Target Big Endian Must Die
- Userland DTrace
- V4L Support in Linux Emulator
- ZFSv28 is Ready for Wider Testing
FreeBSD Team Reports
- FreeBSD Bugbusting Team
- FreeBSD KDE Team
- FreeBSD Release Engineering Team
- The FreeBSD Foundation Status Report
- Enhancing the FreeBSD TCP Implementation
- Five New TCP Congestion Control Algorithms for FreeBSD
- Syncing pf(4) with OpenBSD 4.5
- mandoc/mdocml — groff Replacement for Rendering Manual Pages in FreeBSD
- The FreeBSD German Documentation Project
- The FreeBSD Japanese Documentation Project
- Web Feeds for UPDATING Files
- FreeBSD Services Control (fsc)
- Updating Base Tools to Accommodate Ports Requirements
- xz Compression for Packages and Log Files
- Bringing up ARM to FreeBSD Tree
- FreeBSD on the Playstation 3
- FreeBSD/mips on Octeon
- FreeBSD/mips Ralink RT3052F/Broadcom BCM5354
- Chromium Web Browser
- OpenAFS Port
- pkg_upgrade (sysutils/bsdadminscripts)
- Ports Collection
- Ports Distfile and WWW Checker
- Valgrind Port
- EuroBSDCon 2010
- EuroBSDCon 2011
- FreeBSD Developer Summit, Karlsruhe
- FreeBSD Developer Summit, meetBSD California 2010
Contact: Adrian Chadd <adrian@FreeBSD.org>
FreeBSD-CURRENT runs on the AR9132 SoC. Minor platform-specific tweaks are needed to use it on a given piece of hardware (eg., where in flash the Ethernet MAC address is stored.) The AR910x wireless MAC/PHY is supported. The only available test platform uses a 2.4GHz radio; 5GHz 11a mode has not been tested. As with other Atheros chipset support in FreeBSD, 11n support is not yet finished. The current development platform is the TP-Link TP-WN1043ND 802.11n wireless bridge/router. It is currently being successfully used as a 11bg access point.
- USB support is currently not functional.
- There is currently no support for the Realtek Gigabit switch/PHY chip. This is being worked on.
Contact: Ivan Voras <ivoras@FreeBSD.org>
pkg_patch is a tool meant to be used with the rest of the pkg_* utilities whose job is to create and apply binary patches to FreeBSD package archives. The SoC project was successfully completed but there are some open issues about the integration of the tool in the FreeBSD system. Some changes are necessary to the port/patch infrastructure to support the "update" mode instead of "remove+add".
- Solve pending issues about the ports install/upgrade workflow, probably within the pkg_install2 effort.
We are still in the beginning of the project since we started it after the summer of code.
- Reading ARM structure.
- Reading MicroC OS.
- Using Qemu to emulate the work.
Contact: Romain Tartière <romain@FreeBSD.org>
The BSD# Project is devoted to porting the Mono .NET framework and applications to the FreeBSD operating system.
Mono 2.8 has been released a few days ago and is already available in the BSD# repository. The update breaks a few ports so the lang/mono update in the FreeBSD ports tree will be delayed until those programs are fixed for a smoother update experience.
Work is in progress to include some long-awaited ports such as deskutils/gnome-do but they require a lot of testing and hacking because they have clearly been designed to run on GNU/Linux and portability has never been a priority (which is quite amusing if you consider portability is the main reason to be for mono).
- If you have some time, test mono ports and send feedback.
- If you have more time, join the BSD# Team! There are many ways to help out!
- Currently low priority, some mono hackers who do not use FreeBSD would be interested in a debug live-image of FreeBSD to help us diagnose and fix bugs more effectively.
Contact: Gábor Páli <pgj@FreeBSD.org>
The purpose of this one-day event is to gather Central European developers of today's open-source BSD systems to popularize their work and their organizations, and to meet each other in the real life. We would also like to motivate potential future developers and users, especially undergraduate university students to work with BSD systems. This year's BSD-Day will be held in Budapest, Hungary at Eötvös Loránd University, Faculty of Informatics on November 20, 2010. Everybody is welcome!
During the previous few months several additions were developed to FreeBSD's built-in SNMP daemon — bsnmpd(1).
First a snmp_wlan(3) module was developed that allows monitoring and configuration of wlan(4) interfaces operating in various modes, including statistics, attached/neighboring station information, MAC access control entries and mesh routing information. The module's code was submitted in SVN and is now a part of the FreeBSD base system.
Next, SNMPv3 authentication and encryption support were added to bsnmplib(3), bsnmpd(1) and bsnmptools (which are available via the ports system currently). The message digest and cipher calculation calls use the implementation of the relevant cryptographic algorithm implementation in OpenSSL's crypto(3) library. bsnmpd(1) may still optionally be compiled without the crypto(3) library, in which case only unauthenticated plain-text SNMPv3 PDUs may be processed.
In addition, a snmp_usm(3) module was developed that is used to configure SNMPv3 users parameters (name, authentication & encryption algorithms used and relevant keys, etc.) into bsnmpd(1) as per RFC 3414.
Finally, a snmp_vacm(3) module was developed that allows configuration of view-based access control as per RFC 3415, and relevant checks are made by bsnmpd(1) that allow or restrict access to specific SNMPv1/SNMPv2 communities or SNMPv3 users to certain MIB subtrees as per the configuration in the snmp_vacm(3) module. If none of the SNMPv3-related modules is loaded, bsnmpd(1) preserves its current behavior with SNMPv1/SNMPv2c PDUs.
This work is being funded by the FreeBSD Foundation.
- Update Wiki Page to reflect latest work and document proper use.
- Finish cleanup and have it reviewed.
- More extensive user testing.
Capsicum is a lightweight OS capability and sandbox framework developed at the University of Cambridge Computer Laboratory, supported by a grant from Google. Capsicum extends the POSIX API, providing several new OS primitives to support object-capability security on UNIX-like operating systems: capabilities, a new sandboxed capability mode for processes, anonymous shared memory objects, process descriptors, and a modified C runtime able to support distributed applications within sandboxes. Capsicum has been prototyped on FreeBSD -CURRENT, with a 8-STABLE backport.
We plan to begin merging the core Capsicum kernel features to FreeBSD -CURRENT in November/December 2010 once a number of known problems have been resolved. Following a KBI analysis, we will consider merging our 8-STABLE backport to Subversion. For the time being, and while APIs stabilise, we plan to distribute the Capsicum libraries via ports. However, simply having the kernel features in place is sufficient to support sandboxing in tcpdump and Chromium.
The Capsicum paper by Robert Watson / Jonathan Anderson (Cambridge) and Ben Laurie / Kris Kennaway (Google) won a best paper award at the 2010 USENIX Security Symposium!
- More aggressively test (and as needed, fix) possible UNIX domain socket garbage collector interactions with Capsicum.
- Using results of our recent model checking analysis of the namei() sandboxing approach, make robustness improvements.
- Merge to FreeBSD -CURRENT in November/December.
- KBI analysis for possible 8-STABLE merge.
- Convert more applications to use Capsicum sandboxing!
Contact: Ruben <firstname.lastname@example.org>
Chromium is a Webkit-based web browser that is largely BSD licensed and was recently committed to ports. It has been working well on FreeBSD and supports new features like HTML 5 video. Newer builds use the Clang compiler, Clang first compiled a non-debug build of Chromium, a very large C++ project, on FreeBSD. This porting effort employs a new hybrid-source model: portions of the latest FreeBSD patches are kept closed for a limited time and new builds are made available only to paying subscribers, while older builds are eventually spun off to ports. Further work remains to port all of Chromium to FreeBSD, I am now porting the task manager to use FreeBSD's libkvm and the ALSA audio backend needs to be ported to OSS. There are other issues listed at the porting summary, contact me if you would like to pitch in.
Contact: Ed Schouten <ed@FreeBSD.org>
Contact: Roman Divacky <rdivacky@FreeBSD.org>
Contact: Brooks Davis <brooks@FreeBSD.org>
Contact: Pawel Worach <email@example.com>
Contact: Dimitry Andric <dim@FreeBSD.org>
We recently imported the 2.8 release of Clang into -CURRENT. This release contains many new features and improvements. The integrated assembler ships with this version, but it is not ready for general use yet.
Since r212979, all necessary changes have been committed to be able to build world with Clang, at least on amd64 and i386. It can also be installed and run, and we are now starting the process of shaking out the inevitable bugs.
Because LLVM and Clang are still being improved continuously, we want to import new versions regularly, approximately every two months, to gain access to new features, bug fixes and performance improvements.
There is also an effort on behalf of the ports people, to make as many ports as possible compile and run properly with Clang. Most of the time, this means fixing the incorrect assumption that gcc is the only existing compiler, but sometimes more complicated issues pop up. Help in this area is greatly appreciated.
- Importing new Clang snapshots fairly regularly (approximately bi-monthly).
- Seeing if Clang can be used to build world for ARM (volunteers and ARM experts wanted).
- Fixing as many ports as possible to build with Clang.
- Running periodical ports exp builds with Clang (on amd64 and i386), for example once a month.
Contact: Max Khon <firstname.lastname@example.org>
The purpose of DAHDI/FreeBSD project is to make it possible to use FreeBSD as a base system for software PBX solutions.
DAHDI (Digium/Asterisk Hardware Device Interface) is an open-source device driver framework and a set of hardware drivers for E1/T1, ISDN digital, and FXO/FXS analog cards . Asterisk is one of the most popular open-source software PBX solutions .
The project includes porting DAHDI framework and hardware drivers for E1/T1, FXO/FXS analog, and ISDN digital cards to FreeBSD. This also includes TDMoE support, software and hardware echo cancellation (Octasic, VPMADT032), and hardware transcoding support (TC400B). The work is ongoing in the official DAHDI SVN repository with the close collaboration with DAHDI folks at Digium.
DAHDI/FreeBSD project is completed. ports/misc/dahdi now contains the most recent DAHDI/FreeBSD version and additional stuff that is not available in DAHDI/FreeBSD SVN repository due to licensing and copyright restrictions (OSLEC echo canceler, experimental zaphfc driver). Experimental sparc64 support is also implemented and is currently being tested.
There is a pile of minor changes in queue that will be handled soon:
- Add ability to run asterisk+dahdi under non-root user account.
- Add support for bri_net_ptmp ISDN signalling to asterisk port and drop old and outdated zaptel+asterisk-bristuff ports.
Periodic merges from DAHDI/Linux SVN will be continued on a regular basis with rolling out new DAHDI/FreeBSD releases (most likely synchronized with DAHDI/Linux releases).
Contact: Lawrence Stewart <lstewart@FreeBSD.org>
All outstanding patches have been committed to -CURRENT after a lengthy review process. It is anticipated to merge all of the project's SIFTR and reassembly queue-related patches from -CURRENT to the stable branches in time for the upcoming 7.4 and 8.2 releases.
EuroBSDCon 2010 happened in Karlsruhe, Germany, with many users, developers, friends, and others. We had many tutorials, and 22 interesting presentations on various topics connected to FreeBSD, OpenBSD, NetBSD, like the new USB stack, jail improvements, Virtual Private Systems, SSH and PGP convergence, ZFS, journaled Soft-Updates, BSD certification, porting to the latest ARM processors, and pc-sysinstall. The event was opened by a keynote speech from Poul-Henning Kamp on software tools and their future, and it was closed by short status reports on different BSD flavors.
Contact: Philip Paeps <philip@FreeBSD.org>
EuroBSDCon is the European technical conference for users and developers on BSD based systems. The EuroBSDCon 2011 conference will be held in the Netherlands from Thursday 6 October 2011 to Sunday 9 October 2011, with tutorials on Thursday and Friday and talks on Saturday and Sunday.
The EuroBSDCon conference is inviting developers and users of BSD based systems to submit innovative and original papers not submitted to other European conferences on BSD-related topics.
Please see the EuroBSDCon 2011 website for more details.
Contact: Warner Losh <imp@FreeBSD.org>
One problem that the project has with its push towards embedded platforms is with the toolchain. The compilers and linkers and such in the current FreeBSD support the architectures generically, but often times silicon vendors produce specialized toolchains to wring the most performance out of their silicon. Right now, it is difficult to compile FreeBSD with these tools, as many manual steps are required to make things 'just so'.
The external toolchain project will leverage some of the work done by the Clang team to support Clang in the base system (breaking the strict dependency on CC=cc (except for the broken intel CC support)). In addition, the orchestration of the build (make buildworld) will change to avoid bootstrapping certain tools, or compiling the compilers at all. In addition, support for using alternate assemblers, linkers, etc., will be added. The work will be done in subversion in projects/xtc (for eXternal Tool Chain).
Contact: Zheng Liu <email@example.com>
This project has two goals: pre-allocation algorithm for ext2fs and ext4 read-only mode. Aim of the pre-allocation algorithm is to implement a reservation window mechanism. This mechanism has been implemented and a patch have been submitted. The aim of ext4 read-only mode is to make it possible to read ext4 file systems in read-only mode when the disk is formatted with default features. Until now it can read data from ext4 file systems with default features in read-only mode. A patch has been submitted a patch to the freebsd-fs mailing list and there is a new kernel module, called ext4fs, is under development for it.
- More testing of the pre-allocation algorithm.
Work has commenced on a newly funded FreeBSD Foundation project to bring six modular TCP congestion control (CC) algorithm implementations (the existing NewReno and five new algorithms: HTCP, CUBIC, Vegas, HD and CHD) to the FreeBSD kernel. See the CAIA 5cc and NewTCP websites for more details on the algorithms.
To support the project's primary deliverable, we will also be incorporating the CAIA modular CC and Khelp frameworks into the FreeBSD kernel, along with the Enhanced Round Trip Time Khelp module.
The project will make a sizable, state-of-the-art contribution to FreeBSD and in certain areas, add completely novel work unavailable in any other operating system known to us.
We anticipate a number of benefits, including vastly improved researcher friendliness, reduced work for TCP oriented vendors of FreeBSD-based appliances, and greater choice for system administrators who operate FreeBSD systems in atypical network scenarios.
Keep an eye on the freebsd-net mailing list for project-related announcements.
The bugbusting team continue work on trying to make the contents of the GNATS PR database cleaner, more accessible and easier for committers to find and resolve PRs, by tagging PRs to indicate the areas involved, and by ensuring that there is sufficient info within each PR to resolve each issue.
July saw the addition of Alexander Best (arundel@) to this bugbusting team, he is helping with the triaging PRs as they come in, creating patches for problems and working with submitters to get the solutions tested, and working through the PR backlog.
Also in July, Gavin Atkinson worked with Hans Petter Selasky on the USB PRs, attempting to go through many of them and determine the status of each of them. As a result, nearly 10% of the USB PRs were determined to be closeable, with many more either being marked as patched already or able to be committed quickly. Several PRs that only affect the old (pre-8.0) USB stack were also identified and marked as such. More work will take place in this area in the future.
August saw us host another bugathon, with an aim of investigating and getting into a committable state several of the PRs with patches. Turnout was not as great as in the past — mainly believed to be due to the short notice, but still several PRs were progressed, with several commits made and several PRs closed.
The number of PRs has held steady over the last three months, with improvements in numbers in some categories (especially usb and bin) being offset by slight increases in others.
Reports continue to be produced from the PR database, all of which can be found from the links above. Committers interested in custom reports are encouraged to discuss requirements with bugmeister@ — we are happy to create new reports where needs are identified.
As always, anybody interested in helping out with the PR queue is welcome to join us in #freebsd-bugbusters on EFnet. We are always looking for additional help, whether your interests lie in triaging incoming PRs, generating patches to resolve existing problems, or simply helping with the database housekeeping (identifying duplicate PRs, ones that have already been resolved, etc). This is a great way of getting more involved with FreeBSD!
- Try to find ways to get more committers helping us with closing PRs that the team has already analyzed.
- Try to get more non-committers involved with the triaging of PRs as they come in, and generating patches to fix reported problems.
Contact: Gábor Páli <pgj@FreeBSD.org>
We were happy to have more than 40 FreeBSD developers and guests attending the FreeBSD Developer Summit prior to EuroBSDCon 2010 in Karlsruhe, Germany. This workshop-style event was hosted at Karlsruhe Institute of Technology, and included prepared presentations in the morning, as well as group hacking and discussion sections in the afternoon. We had various talks on several topics, covering the USB subsystem, state of the toolchain, the FreeBSD documentation, NanoBSD improvements, FreeBSD port of PF, jails, Virtual Private Systems, cooperation with the PC-BSD Project, FreeNAS, the new event timers subsystems, bugbusting discussions and Ports Tinderbox presentations, and many of this year's and last year's Google Summer of Code projects. Photos, videos, and slides for most of the talks are available on the wiki page.
Contact: Warner Losh <firstname.lastname@example.org>
We will be having a developers summit meeting at meetBSD California 2010 on November 4th, the day before the conference. Based on who is in attendance, we will be talking about the status of pressing issues; working on pressing problems and using the opportunity for face to face meetings to work out issues that are difficult in email. This is an invitation-only event, but any developer can invite people they think would help drive this meeting forward. An agenda will be published closer to the date.
Contact: FreeBSD KDE Team <kde@FreeBSD.org>
Contact: Thomas Abthorpe <tabthorpe@FreeBSD.org>
Contact: Max Brazhnikov <makc@FreeBSD.org>
Contact: Kris Moore <kmoore@FreeBSD.org>
Contact: Dima Panov <fluffy@FreeBSD.org>
Contact: Alberto Villa <avilla@FreeBSD.org>
The FreeBSD KDE team has been actively keeping pace with development cycle as it is released by the KDE developers. Often having KDE in the ports tree within the same week it has been released.
We are fortunate to have a strong contributor base that helps to keep the process moving along. Our heartfelt thanks go out to all that have helped with patches, maintaining ports, and responding with help on the mailing lists.
- KDE 4.5.4 is due out at the end of November, with 4.6.0 to be released early in 2011.
- The FreeBSD KDE team is always looking for helpers, if you are interested in assisting, please feel free to contact any of our team members.
FreeBSD/powerpc64 now boots multi-user SMP and is self-hosting on the Playstation 3. Booting requires a PS3 console with the OtherOS capability (fat model console with firmware < 3.21). The only supported hardware at present is USB and the Ethernet controller.
- SATA support.
- Boot loader enhancements to allow user input at the loader prompt.
- Support for the Cell SPU units.
Contact: Release Engineering Team <re@FreeBSD.org>
The Release Engineering Team has announced the schedule for the upcoming joint release of FreeBSD 7.4 and 8.2. The schedules are available on the web site:
It is expected that 7.4 will be the last of the 7.X releases.
Contact: Tom Rhodes <trhodes@FreeBSD.org>
FreeBSD Services Control is a mix of binaries which integrate into the rc.d system and provide for service (daemon) monitoring. It knows about signals, pidfiles, and uses very little resources.
The fsc daemon (fscd) runs in the background once the system has started. Services are then added to this daemon via the fscadm control utility and from there they will be monitored. When they die, depending on the reason, they will be restarted. Certain signals may be ignored (list not decided), and fscd will remove that service from monitoring. Every action is logged to the system logging daemon. Additionally, the fscadm utility may be used to inquire about what services are monitored, their pidfile location, and current process id.
FSC provides several advantages over the third-party daemontools package. For example, fscd uses push notifications instead of polling; fscd is an internal, FreeBSD-maintained software package accessible to all developers where daemontools would have to be a port and require us to maintain patches; fscd could be easily integrated with the current rc.d infrastructure.
Partially based on the ideas of daemontools and Solaris Service Management Facility (SMF), this could be an extremely useful tool for FreeBSD systems.
Since the last status report, two bugs have been fixed and the documentation has been updated. In the coming weeks we hope to get more developer attention and review, perhaps even push to commit the code into FreeBSD.
- Testing and feedback would be really helpful.
Contact: Juli Mallett <jmallett@FreeBSD.org>
All Octeon development is now ongoing in -CURRENT and most Octeon-specific and general MIPS changes from the old Octeon branch have been checked in. The Simple Executive from the Cavium Octeon SDK has been checked into Subversion and most of the Octeon port has been updated to use it where appropriate, including moving to a port of the Linux Ethernet driver, octe. SMP support is stable on 2-core systems and has seen some testing on systems with up to 16 cores.
- Some PCI devices still do not seem to work completely.
- Host-mode USB support is incomplete and needs further testing and debugging.
- Work on an ATA-based Compact Flash driver for boards that support DMA has begun.
- A GPIO driver should be trivial using the Simple Executive.
- Performance in the Linux-derived octe Ethernet driver could be improved. Support for some switch chipsets that are commonly present in Octeon-based equipment is in progress.
Contact: Aleksandr Rybalko <email@example.com>
FreeBSD/mips has been ported to D-Link DAP-1350, wireless AP/router based on Ralink RT3052F SoC.
- rt2860: Ralink RT2860 802.11n — Worked, but RT3022 2.4G 2T2R radio tuning required.
- rt: Ralink RT3052F onChip Ethernet MAC — Done.
- rtsw: OnChip Ethernet switch — Not done (initialized by UBoot).
- usb-otg: DWC like USB OTG controller — Worked.
- gpio: RT3052F onChip GPIO — Worked (LEDs, Buttons).
- cfi: CFI NOR Flash — Worked.
FreeBSD/mips D-Link DIR-320 project(BCM5354 SoC).
New profile openvpn-router available for testing.
- Debug/Fix USB OTG driver (RT3052F).
- Debug/Fix 802.11n driver (RT3052F).
- Write rtswitch driver (RT3052F).
- Implement Timer unit driver (RT3052F).
- Implement Hardware NAT/PPPoE/VLAN offload (RT3052F).
- Implement I2C/I2S/PCM/SPI drivers (RT3052F).
- switch configuration utility (BCM5354).
Contact: Marius Strobl <marius@FreeBSD.org>
Apart from the constant bug fixing and adaptions to machine-independent changes that pretty much always take place, not much has happened in the area of sparc64 since the last status report. The only noteworthy exception are some performance optimizations which take advantage of features of Fujitsu SPARC64 CPUs. These were a bit too risky for putting them in shortly before FreeBSD 8.1-RELEASE but will be part of 7.4-RELEASE and 8.2-RELEASE now that they have received the necessary testing.
Part of reasons why not much has happened in this spot was some lack of time on my side but also due to nobody showing up with a not yet supported sun4u machine lately and me delving in the network land instead, which yielded some things to report about in the next status report. On the other hand I recently got a hold of a Sun Fire 3800, so these and other models from the same family likely will be supported by FreeBSD at some point in the future.
Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
There are three new GELI (a disk encryption GEOM class) features available in -CURRENT:
- AES-XTS encryption. XTS mode is a standard that is recommended these days for storage encryption. This is the default now. AES-XTS support was also added to opencrypto framework and aesni(4) driver.
- Multiple encryption keys. GELI will use one encryption key for at most 2^20 blocks (sectors), as it is not recommended to use the same encryption key for too much data. It generates a key array from the master key on attach and uses it accordingly. This is the default now.
- Passphrase can now also be loaded from a file (-J and -j options).
Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
The gptboot now fully follows GPT specification (verifies checksums and falls back to backup header and table if primary is corrupted).
One can now use new attributes to configure partition that gptboot will try to boot only once from and in case of a failure it will fall back to the previous one.
For more information check out the commit message.
Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
HAST is now better than ever! Some recent improvements include:
- Hooks supports — HAST will execute the given command on various events (connect, disconnect, synchronization start, synchronization completed, synchronization interrupted, split-brain condition, role change).
- Configuration reload on SIGHUP, a very missing functionality.
- Internal keepalive mechanism.
- Many bug fixes, majority of them reported by Mikolaj Golub.
Contact: Alexander Motin <mav@FreeBSD.org>
Work on new event timers infrastructure continues. In -CURRENT amd64, arm (Marvell), i386, mips, pc98, powerpc, sparc64, sun4v architectures were refactored to use new timers API.
New machine-independent timers management code was written. It can utilize both legacy periodic and new one-shot timer operation modes.
Using one-shot mode allows to significantly reduce the number of timer interrupts and respectively increase CPU sleep time during idle periods. Timer interrupts on idle CPUs are now generated only when they are needed to handle registered time-based events. Busy CPUs unluckily still receive the full interrupt rate for purposes of resource accounting, scheduling and timekeeping.
With some additional tuning it is now possible to have an 8-core system, receiving only about 100 interrupts per second and respectively have CPU idle periods up to 100ms. This allows to effectively use any supported CPU idle states (C-states), that reduces power consumption and increases effect of the Intel TurboBoost technology.
New manual pages were written to document this functionality: eventtimers(7), attimer(4), atrtc(4), hpet(4).
- Troubleshoot possible hardware issues.
- Refactor remaining architectures (arm, ia64, XEN PV).
- Do some optimizations in different subsystems to reduce number of time-based events. Extend callout API with terms of precision, allowing to group close events.
- Make schedulers tickless, or at least less depending on time events to make skipping timer interrupts possible when CPUs are busy.
- Merge code into 8-STABLE when it is considered ready.
Contact: Gleb Kurtsou <gk@FreeBSD.org>
PEFS is a kernel level stacked cryptographic file system, i.e. it stacks on top of existing mounted filesystems. AES and Camellia algorithms in XTS mode are supported. The project has matured since Summer of Code 2009, most important improvements for last few months include: switch to use XTS encryption mode, implementation of sparse file support, fixing rename bugs including race and livelock conditions, addition of ext2 support. PEFS suite contains pam module facilitating user authentication with file system key and adding keys to mounted file system on login. PEFS passes fsx, pjdfstest, blogbench and dbench tests running on top of UFS and ZFS.
Contact: Ulrich Spörlein <uqs@FreeBSD.org>
Kristaps' groff-replacement (only for rendering manual pages) is already available in NetBSD and OpenBSD, and used to render the base system manpages for the latter. This project aims to do similar things for FreeBSD.
mandoc(1) is more strict in what it accepts as input and is still lacking some features that are used by some selected few manpages.
Getting manual page fixes accepted by upstream vendors has been challenging. Waiting for them to round-trip back into FreeBSD will take even longer. Future work will therefore result in direct commits to our contrib/ and gnu/ repository areas, in the hope this will not impact future vendor imports too much.
- Finish the Big Manpage Cleanup of 2010.
- Write a textproc/groff port for the latest groff version.
- Import mandoc(1), switch to catpages for base.
- Supply necessary ports infrastructure to opt-in to mandoc(1).
- Discuss future of groff(1) in base wrt. share/doc.
Netdump provides kernel core dumping over the network, instead of to a local disk. It implements a very minimal TCP/IPv4 stack and uses a custom UDP protocol to transmit the dump to the netdump server running on another host. Network interfaces selected for dumping perform I/O in polling mode.
Netdump should find its use in diskless workstation clusters, PXE-booted test machines, and perhaps when doing disk driver development.
- General FreeBSD dumping mechanism refinements.
- Implement checksum on UDP packets.
- Investigate the possibility to replace the custom protocol with tftp.
- Investigate the possibility to replace the custom TCP/IPv4 stack with Contiki.
- Implement network console and gdb backend using a shared debug context stack.
- Add IPv6 support.
AFS is a distributed network file system that originated from the Andrew Project at Carnegie-Mellon University; the OpenAFS client implementation has not been particularly useful on FreeBSD since the FreeBSD 4.X releases. The previous status report brought the OpenAFS client to a useful form on -CURRENT, though with many rough edges. Only a couple of those edges have been smoothed out during the past few months, as developer time was scarce. A mismatch between file size and vmobject size tracking was resolved (allowing executables to be run from AFS), and our system call entry has been updated on -CURRENT and 8-STABLE to match reality. Thanks to Kostik Belusov for both of those! The code is useful enough that we plan to submit an openafs-devel port to the Ports Collection in the coming cycle.
There are several known outstanding issues that are being worked on, but detailed bug reports are welcome at firstname.lastname@example.org.
- Rework vnode locking for lookup operations to avoid an easily-triggered deadlock between two threads when one is looking up the parent directory.
- Update VFS locking to allow the use of disk-based client caches as well as memory-based caches.
- Track down races and deadlocks that appear under load.
- Integrate with the bsd.kmod.mk kernel-module build infrastructure.
Contact: Alexander Fiveg <afiveg@FreeBSD.org>
Ringmap is a complete FreeBSD packet capturing stack specialized for very high-speed networks. The goal of this project is to develop the software for efficient packet capturing and integrate it with the generic network drivers and libpcap.
- Integrated with the lem driver. Intel network controllers: 8254X are supported.
- Packet filtering using BPF in both kernel and user space.
- Partly integrated with ixgbe driver for 10Gb capturing.
- Support for hardware timestamping.
- Writing packets to the disc from within the kernel.
- Multiqueue support.
- Extending the "ringmap" for packet transmission.
Contact: Kris Moore <kmoore@FreeBSD.org>
Work is progressing quickly on a major re-factoring of PC-BSD tools and the PBI format for 9.0. Our GUI tools have been converted to compile / run within native QT without KDE now, allowing us to begin offering support for other desktop environments for 9.0, such as Gnome, XFCE, LXDE, KDE, etc. The PBI format has undergone a complete evolution, and is now entirely command-line based for all aspects of it, with only a few dependencies upon curl & xdg-utils. This will allow us to begin offering PBIs for traditional FreeBSD users starting with 9.0, who will be able to install the pbi-manager from ports in the near future.
- We are still busy converting / fixing all our tools to play nicely with various DE's, but making quick progress.
- The new PBI format is still undergoing extensive testing, and bugs are being isolated and fixed.
pc-sysinstall was imported into CURRENT recently. For the moment it is feature complete, although progress on the text front end for it may expose additional functionality it needs.
- The automated/scripted install features of pc-sysinstall need wider testing and use to expose potential weaknesses, bugs, and additional features it may require.
- Related tasks include getting a text front-end to pc-sysinstall working and hooking up pc-sysinstall to the build so install media is generated that runs pc-sysinstall.
Contact: Dominic Fandrey <email@example.com>
pkg_upgrade was (to my knowledge) the first binary packages only update tool for the FreeBSD ports. Using it does not require a copy of the ports tree.
Currently the tool is in the final stages of a recode, that will greatly improve support for sharing packages over NFS or nullfs mounts (e.g. for distributing packages into jails) and also offers improved dependency tracking and performance, more in line with how pointyhat and Tinderbox build packages.
I recently had the opportunity to present my work at the EuroBSDCon 2010.
- Complete session code.
- Add INDEX generator script that harvests information directly from packages and hence is always accurate.
The ports tree count now exceeds 22,000. With the assistance of many people, especially Philip Gollucci, the open PR count is below 1000 for the first time in quite a while. This is very encouraging progress.
Since the last report, we added five new committers, and took in two commit bits for safe keeping.
With onsite assistance from jhb@, gnn@, skreuzer@, and pgollucci@, we now have 11 new servers at NYI. The machines still need testing for stability and will soon be assigned for package building.
The Ports Management team have been running -exp runs on an on-going basis, verifying how base system updates may affect the ports tree, as well as providing QA runs for major ports updates. Of note, -exp runs were done for:
- des: test libfetch
- gabor: tests for BSD iconv and grep
- mezz: switch www/neon28 to www/neon29
- beat: update www/libxul
- johans: update devel/bison and devel/m4
- dinoex: update graphics/tiff
- jpaetzel: update devel/popt
- ade: multiple runs autotools upgrade
- gerald: setting USE_GCC=4.5 as default
- ashish: changes to Mk/bsd.license.mk
- kwm: test of Clang in -CURRENT
- Looking for help fixing ports broken on -CURRENT.
- Looking for help with Tier-2 architectures.
- Most ports PRs are assigned, we now need to focus on testing, committing and closing.
Contact: Emanuel Haupt <ehaupt@FreeBSD.org>
Given the current status of fenner's Distfiles Survey, a new distfile checker was written in order to have an overview for the state of each distfile in the ports tree. The distfile checker is also able to verify WWW entries in pkg-descr files. This is an attempt to weed out broken MASTER_SITES and outdated WWW entries.
The current version uses a MySQL database backend and is able to verify 432512 distfiles (30 concurrent threads) within 24 hours.
Contact: Ilya Bakulin <kibab@FreeBSD.org>
All work is now in Perforce. Rich set of features is added to the kernel, userland tools and libc modifications are ready, documentation is ready.
- Documentation review.
- Presentation of feature set on the various mailing lists.
- Committing to -CURRENT, possibly merging to stable branches (changes do not break ABI/KBI).
Contact: Edward Tomasz Napierala <trasz@FreeBSD.org>
The goal of this project is to implement resource containers and a simple per-jail resource limits mechanism. Resource containers are also a prerequisite for other resource management mechanisms, such as Hierarchical Resource Limits, for "Collective Limits on Set of Processes (aka. Jobs)" Google Summer of Code 2010 project, for implementing mechanism similar to Linux cgroups, and might be also used to e.g. provide precise resource usage accounting for administrative or billing purposes. So far, a generic resource usage framework has been developed, along with limit enforcement for most resources. Work is on-going on adding limits for remaining resources, debugging and generally improving the implementation. This project is being sponsored by The FreeBSD Foundation.
Contact: Ermal Luçi <eri@FreeBSD.org>
This work is based on OpenBSD 4.5 state of pf(4). It includes many improvements over the code currently present in FreeBSD. The actual new feature present in pf45 repository is support for divert(4), which should allow tools like snort_inline to work with pf(4) too. This work also enables pfsync(4) to be loaded as a module as well.
Currently, this work is considered stable and a patch against -CURRENT has been released on freebsd-pf mailing list.
The reason why this work is based off of OpenBSD 4.5 is that after this release they have changed the syntax which is not backwards compatible.
After importing this one the work will go on the newest version and decisions on it will then be done.
- Make a decision whether we need pflow(4) in base.
- More regression testing is needed.
Contact: Warner Losh <imp@FreeBSD.org>
The "tbemd" or Target Big Endian Must Die effort is nearing completion. Most of the big sweeping changes to the tree have been committed. The last change, actually pulling the switch, is stalled waiting for make universe improvements. This work will change the TARGET_ARCH from a plain 'mips' to 'mipsel' or 'mipseb' based on which endian the platform has. It introduces the concept of multiple architectures being implemented with one set of files, and regularizes that design pattern into the FreeBSD build process. In the past, you had to set TARGET_BIG_ENDIAN=t to compile for big endian, but that had a number of problems: can not share /usr/obj between little and big endian targets, sometimes the produced compilers will not work right unless TARGET_BIG_ENDIAN is defined in the environment, etc.
- Update make universe to cope with the new architectures when building kernels.
Contact: Deb Goodkin <deb@FreeBSDFoundation.org>
We were proud to be a sponsor for MeetBSD 2010 Poland and KyivBSD 2010 in Kiev, Ukraine. We also committed to sponsoring BSDDay Argentina 2010, MeetBSD California 2010, and NYBSDCon 2010 all in November. The Foundation was also represented at MeetBSD Poland and Ohio LinuxFest.
Completed the Foundation funded projects: "FreeBSD Jail-Based Virtualization" by Bjoern Zeeb and "DTrace Userland" by Rui Paulo.
We kicked off a new project by Swinburne University called "Five New TCP Congestion Control Algorithms for FreeBSD".
We continued our work on infrastructure projects to beef up hardware for package-building, network-testing, etc. This includes purchasing equipment as well as managing equipment donations.
We are three quarters of the way through the year and we have raised around $160,000 towards our goal of $350,000. Find out how to make a donation at http://www.FreeBSDFoundation.org/donate/
Stop by and visit with us at MeetBSD California (Nov 5-6), LISA (Nov 10-11), and NYCBSDCon (Nov 12-14).
The committers to the German Documentation Project were mostly trying to keep the documents and the website translations in sync with the ones on FreeBSD.org. Fabian Ruch was helpful in catching up with the changes to the Porters Handbook. Benedict translated the Solid State article into German because this is becoming a good addition to traditional hard drive storage.
We tried to re-activate committers who did not contribute for some time but most of them are currently unable to free up enough time. We hope to gain fresh contributor blood as we are getting occasional reports about bugs and grammar in the German translation.
- Submit grammar, spelling or other errors you find in the German documents and the website.
- Translate more articles and other open handbook sections.
The www/ja and doc/ja_JP.eucJP/ have been updated constantly since the last status report. We committed a big patch for the "Installing FreeBSD" chapter of the FreeBSD Handbook which was contributed by many people since a long time. This chapter is still outdated and needs more work. Some progress was made in the Porter's Handbook as well.
- Further translation of the FreeBSD Handbook and contents of the www.FreeBSD.org site to the Japanese language.
- Pre-/post-commit review of the translation.
Contact: Gordon Tetlow <gordon@FreeBSD.org>
The goal of the project is to allow easier extension of base system tools by the ports system. Ideally, no files in /etc should need to be modified by a port installation.
The man toolset was recently reimplemented as a BSDL version instead of the old GPL version. It is also a single shell script instead of multiple C programs. Ports can extend the man functionality by dropping files into /usr/local/etc/man.d/portname.conf.
Next up on the list is to finish the implementation for newsyslog thereby allowing ports that need logs rotated to take advantage of that tool.
Contact: Hans Petter Selasky <hselasky@FreeBSD.org>
During the last two months the USB stack in -CURRENT has been enhanced to support USB 3.0 and the XHCI USB 3.0 chipset from Intel. The XHCI chip will eventually replace the EHCI, OHCI and UHCI chips.
- FreeBSD testers which have access to USB 3.0 hardware are wanted.
Contact: Rui Paulo <rpaulo@FreeBSD.org>
Userland DTrace support was a FreeBSD Foundation sponsored project that was developed during this summer. The project aimed to bring the userland DTracing functionality to FreeBSD as it is available on OpenSolaris. FreeBSD now supports the pid provider and the usdt probes. plockstat is available with a separate patch. Dtruss, a DTrace script that works similarly to ktrace, but with other advantages was imported into FreeBSD. The mysql-server and postgresql-server ports also have DTrace support.
Contact: J.R. Oldroyd <firstname.lastname@example.org>
The V4L support in the Linux emulator has been merged to 8-STABLE allowing use of video in Skype calls using a camera supported by the pwcbsd or video4bsd drivers. A known issue for Skype is that your camera must support YUV420 mode which is what Skype uses. Note that V4L2 support is not included in the current work, and remains as a project for anyone interested.
Valgrind is a tool for detecting memory management and threading bugs, and profiling. Version 3.6.0 has recently been released and the FreeBSD port has now been updated.
Development of the Valgrind port has moved from Perforce to bitbucket.org, in order to make it easier for others to track changes as we progress towards getting the port into shape to commit upstream. The repository's Bitbucket address is at the beginning of the report.
A bugzilla entry has been submitted to track the FreeBSD Valgrind port. You can see the status and vote for the bug to express your interest at https://bugs.kde.org/show_bug.cgi?id=208531.
- Port exp-ptrcheck valgrind tool and fix outstanding issues that show up in memcheck/helgrind/DRD in the Valgrind regression tests suite.
- More testing (please, help).
- Integrate our patches upstream.
Contact: Alexander Kojevnikov <email@example.com>
updating.versia.com features web feeds for UPDATING files from ports, head, stable/7 and stable/8. These feeds provide an easy way to track important changes in the ports tree and the base system.
Contact: Martin Matuska <mm@FreeBSD.org>
Support for xz compression has been enabled in bsdtar (-CURRENT 8-STABLE) and added to pkg_create(1) and pkg_add(1) (-CURRRENT). Packages with the .txz suffix can be created and installed. Log file compression using xz in newsyslog(8) will be integrated soon. Benchmarks show 15-30% better compression ratios and up to halved decompression times when compared to bzip2. A switch from the default package format from .tbz to .txz is to be considered.
- Test building all FreeBSD packages with xz compression.
Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
ZFS v28 which includes data deduplication and plenty of other shiny new features is ready for testing. For more information check out the announcement.