daapd, a server for the DAA protocol
version 0.2.4
 
(c) deleet 2003, 2004, Alexander Oberdoerster 
(http://www.deleet.de/projekte/daap/daapd/)

(see bottom of file for license)


PURPOSE

daapd scans a directory for music files and makes them available via the Apple proprietary protocol DAAP. DAAP clients can browse the directory and retrieve individual files, either by streaming or by downloading them. 


ABOUT DAAP

DAAP is an http-based protocol for serving music content (and probably other media, too) over TCP/IP networks. DAAP was established by Apple iTunes 4.0 in April 2003. Official documentation is not available to date. 

Further information on DAAP:
DAAP wiki                   (http://www.deleet.de/projekte/daap/)
DAAP mailing list           (mailto://daap-dev-subscribe@develooper.com to join)
Todd Larason's DAAP Notes   (http://molelog.molehill.org/blox/Computers/Macintosh/DAAP3.writeback)
DAAP documentation          (http://daap.sourceforge.net/docs/)
Another DAAP documentation  (http://tapjam.net/daap/)


INSTALLATION

First of all, if you have problems with the installation, read the FAQ. A few common problems are outlined there.

The following libraries have to be installed prior to daapd installation:

libid3tag            (part of MAD, http://www.underbit.com/products/mad/)
libz                 (http://www.zlib.org)

The following libraries are optional:

howl                 (for Zeroconf/Rendezvous, http://www.porchdogsoft.com/products/howl/)
mpeg4ip              (for AAC metadata, http://mpeg4ip.sourceforge.net/)

Please see their respective documentation for their installation process. 

In case of MAD, you only need libid3tag. It is distributed separately. 
In case of mpeg4ip, you only need mp4v2. See README.mpeg4ip.
If you want to use howl, the mDNSResponder included with howl has to be started before daapd. Just launch mDNSResponder with no arguments. See also README.howl. 

Installation of daapd itself is straightforward: Simply decompress the distribution .tar (which you have already done, since you are reading this file) and run 'make' in the distribution directory. Making only works with GNU Make.

If you have not installed howl or mpeg4ip, disable them in the makefile.

If you have installed any of the libraries and/or the include files to a non-standard location, just add this location to the makefile. 

daapd has been tested on the following systems:

MacOS X 10.3.7 (gcc 3.3)
Linux kernel 2.4.25 (i386, gcc 3.3.2)
FreeBSD 4.9-RELEASE (i386, gcc 2.95.4)

but it should compile and run on most POSIX systems with minimal changes. Please contact the author if you manage to run daapd on systems not listed here.


USAGE

daapd has the following invocation syntax (see also the daapd man page):

daapd [-dhqvz] [-c config-file] [-C cache-file] [-n name] [-p port] [-t vbr-limit] [file/directory]...

When called without command line arguments, daapd looks for a configuration file called /etc/daapd.conf. Failing that, it runs with the following default options:

- current directory is scanned for music files
- port is 3689 (port registered for DAAP with IANA)
- server name is hostname
- database name is hostname
- no password
- no cache
- check for deleted items
- don't output verbose information
- parse all files for length; parse all mp3 frames
- compress transmissions with gzip

The command line option '-c' specifies an alternative configuration file, '-C' specifies the cache file (see below), '-n' specifies both server and database name, '-p' specifies the port daapd runs on.

The option '-t' is best left alone in most cases. If you get incorrect durations for a few songs or you can do without duration times in exchange for a faster startup, read the section PERFORMANCE.

With option '-v' daapd produces extensive debugging information, with '-q' daapd produces no output at all. Option '-d' activates daemon mode: daapd detaches itself from the terminal after launch. '-d' implies '-q'.

The option '-z' turns off compression. This is useful on slow machines with fast connections.

Options in the configuration file override the defaults. Command line options override both the defaults and options in the configuration file.

An arbitrary number of directories and files (to the length of the command line) can be supplied on the command line. daapd searches the directories recursively for music files.

The scan is performed after daapd is invoked. It may take a few minutes, depending on CPU and disk speed and the size of your music library. After this process is finished, daapd is ready for accepting client connections.

If a cache file is specified, daapd attempts to read it prior to the directory scan. Files that are already in the cache and that are current (have not been changed since the cache file was created) are not re-scanned. This significantly reduces startup time. After the directory scan, the cache file is updated with the changed data. If the cache file is not present, it is created.

In iTunes version 4.0, there was an option to connect to a specific host specified by IP or hostname. As of Version 4.0.1, this option is gone. iTunes 4.0.1 and later can only connect to servers which advertise themselves via Rendezvous/zeroconf. daapd uses the howl library to do so if howl is enabled in the makefile. Howl is enabled by default.

If howl is not supported on your architecture, you have to use an external application, for example Apple's mDNSResponderPosix. Launch the responder on the same host as daapd with the following options:

mDNSResponderPosix -p 3689 -t _daap._tcp. -n `hostname`

By setting up a mDNS proxy, you can also connect iTunes to a remote daapd. The proxy has to be set up on a host that's located in the the same subnet as the iTunes client. You can use mDNSProxyResponderPosix for that. The syntax is:

mDNSProxyResponderPosix REMOTE-IP HOST-LABEL SERVER-NAME _daap._tcp. 3689 

Where REMOTE-IP is the internet address of the server daapd runs on (can be an IP as well as a DNS name) and SERVER-NAME is the desired name that shows up in iTunes before the connection is established. HOST-LABEL is irrelevant in this context and can be set something arbitrary like 'foo', but has to be present.

You can get both mDNSResponderPosix and mDNSProxyResponderPosix from http://developer.apple.com/darwin/projects/rendezvous.

If you prefer a GUI tool, have a look at Rendezvous Beacon (MacOS X only): http://www.chaoticsoftware.com/ProductPages/RendezvousBeacon.html


CONFIGURATION FILE

The configuration file consists of one variable per line. One variable is specified as follows:

VarName Value

The VarName is case-sensitive. daapd recognizes the following VarNames:

Port		- tcp port daapd accepts connections on
ServerName 	- name of the server displayed in the connection dialog
DBName		- name of the database
Password	- http authentication password (username is ignored, as iTunes only asks for password) 
Root		- directory searched for music files
Cache		- full path of the cache file; empty string means no cache
Timescan	- see PERFORMANCE
Compress	- gzip compression of transmissions; 0 for off, 1 for on

See also daapd-example.conf.


PERFORMANCE

(This section has been obsoleted with the support of Xing/LAME headers in daapd 0.2.3b. Calculating the duration from these headers is fast. Most VBR MP3 files include such a header.)

The first daapd startup can take a lot of time (or any startup, if you're not using a cache). Most of the time is spent with parsing the mp3 files for their exact duration. Why is this necessary? And why does this take so long?

MP3 files are divided in frames. Each frame has a header that contains, among other things, the bit rate and the sampling rate for that frame. Because each frame can have a different bit rate and sampling rate, but the duration (in milliseconds) of a frame is fixed (13 or 26 ms, depending on a few things), the length (in bytes) of a frame is not constant. 

So, in order to determine the duration of a mp3 file, the frames have to be counted. To do this, each frame header has to be read and decoded. Only with the decoded header can the start of the next frame be determined. File operations like these are very expensive (even with memory mapped files), and this adds up with a big mp3 collection. In effect, the whole process is not very unlike reading the complete mp3 collection from disk (probably 30 GB or more).

Unfortunately, there is no reliable way around this. As far as I know, at least. If you do know a better way, please drop me a mail. 

But there are methods that work most of the time and speed up the process somewhat. 

First, there is an ID3 frame for the track duration. If a file contains this frame, daapd is happy and doesn't parse it at all.

Second, there are mp3 files with constant bit rate throughout the whole file (Constant Bit Rate or CBR) in contrast to variable bit rate (VBR). There is no reliable way to tell a CBR file from a VBR file; in principle, the bit rate could be constant thoughout most of the file and change only in the very last frame. But in practice, the bit rate changes already in the second frame in almost all files. So daapd is comparing the first to frames and stops parsing when their bit rates are equal.

If you have VBR files that change bit rate later, daapd will calculate a wrong duration for these files. If that is the case, you can tweak daapd with the '-t' option (or the Timescan variable).

In the option '-t vbr-limit', vbr-limit has the following possible settings:
-1: mp3 files are not parsed for duration at all.
 0: all mp3 files are assumed to be VBR and parsed entirely. 
>0: if the first n frames match, the file is assumed to be CBR and parsing is stopped.

-1 is fastest, but most songs will have a 'continuous' (undefined) duration in iTunes; only songs with duration information in the ID3 header will have a defined duration. With positive values, daapd will only parse the first few mp3 frames. 0 is slowest, but will yield a correct duration for any file. 


TO DO

In order of importance/interest:

- on-the-fly re-coding of File formats iTunes doesn't support
- web admin interface
- support persistent ids and the /resolve request


KNOWN ISSUES

- The initial file scan is slow. It was a conscious design decision for daapd to run transparently on the filesystem and not to require the use of a database. Caching the gathered data speeds up this process (option '-C').

- The filesystem is periodically re-scanned (polling). Updates without polling (interrupt-driven) could be implemented with SGI FAM (http://oss.sgi.com/projects/fam/) or similar OS-specific mechanisms. 

- The password is stored as plain text in the configuration files. This is an obvious security issue. daapd is probably full of other security holes as well.


VERSION HISTORY

0.2.4

- static playlists. daapd reads playlists in M3U, M3U extended and PLS format. daapd adds any playlist it finds during the file scan to the list of playlists and adds all included files to the database. Files not found are not added; relative paths are resolved. The order of the titles in playlists is preserved. daapd also forwards URLs to iTunes, so playlist entries can point to other servers or web radio stations. Only the http protocol is supported. AFAIK, this is a limitation of iTunes. 

There are still some issues with playlists: 

- Auto-update does not work correctly when playlists are deleted or modified (adding playlists works).
- Title and length info (M3U Extended, PLS) is not parsed yet. This is only relevant for tagless files and URLs.
- Paths with backslashes are not parsed correctly.

0.2.3d

- fixed a stupid bug in the howl callback. If you had problems with daapd showing up in iTunes (i.e. with the mDNS Responder), try this update. Thanks to Bruce Sheplan for this one.

0.2.3c

- the howl API has changed as of howl 0.9.6. daapd 0.2.3c reflects this (and is only compatible with this version).

0.2.3b

- added support for Xing/LAME headers. This speeds up file parsing (duration calculation in particular).
- fixed a few memory leaks.
- fixed an issue in the makefile.

0.2.3a

- added auto-update-support. If the periodical scan finds changes in the filesystem, it notifies all connected clients waiting for an update. At the moment, the complete database is sent; sending just a delta is forthcoming.

0.2.3

- daapd scans the filesystem periodically every 5 minutes now. This happens in a separate thread.
- daapd compresses the database with gzip now. This can be turned off with -z (useful on slow machines and fast connections).

0.2.2c

- changed libhttpd to use a smaller buffer. This reduces memory requirements drastically with large files and leads to the stream starting faster.

0.2.2b

- fixed a bug in the mp3 duration calculation routines. Under certain circumstances, this caused iTunes to chop off tracks at the end. Thanks to Yann Bizeul for pointing this out.

0.2.2a

- changed a few things to make daap compatible with iTunes 4.5, including a few changes in libhttpd, so make sure it is rebuilt as well. 
- added support for encoded AAC files: daapd streams .m4p files now, but the client has to be authorized to play them.
- added a stub for a future admin interface. Pointing a web-browser to "<server-ip>:3689/admin/rescan" causes daapd to rescan the database. 

0.2.2

- added support for MPEG4/AAC metadata (thanks to Robert Böhrs and Stephen Lee).
- added service advertisement (DNS-SD) via howl (thanks to Sandy McArthur for the hint).
- added a daemon mode (thanks to Dirk-Willem van Gulik).
- fixed a bug that files that appear more than once in a directory structure get added multiple times. daapd ignores duplicates now (thanks to Mikol Graves).
- added an rc script for linux/sysv init to the distribution (thanks to Mykle Hansen)
- daaplib and libhttpd-persistent are now included in the distribution

0.2.1d

- renamed libhttpd to libhttpd-persistent to allow both versions to be installed at the same time. This became neccessary because libhttpd-persistent turned out to be incompatible to the old version.
- added support for comments / the ID3 comment field  (thanks to Remi Gurski). If you have problems compiling, check that you have libid3tag >= 0.15.0b installed. 

0.2.1c

- automatic rescan of all the files on client connect is now off, because daapd turned out to be unresponsive during this process so all existing client connections are disrupted. Added an option for turning it back on.

0.2.1b

- fixed a file descriptor leak (thanks to Ronald Rael Harvest).

0.2.1

- fixed a bug that caused daapd to segfault when encountering malformed id3 headers.
- fixed a bug that caused daapd to hang when encountering special files (character devices and the like)
- fixed a bug that caused to stop scanning and fail to open the http server with big directory structures (forgot to close directories after opening them)
- fixed a bug in libhttpd that caused daapd to consume unnecessary amounts of CPU when transmitting files
- fixed a bug that caused to songs to appear multiple times. This only happened on Linux systems reading from VFAT file systems. (The cause was the fact that the Linux implementation of VFAT does not supply stable inodes. as the song database was indexed by inode, songs were not found in subsequent scans and thus added although they were already present.)
- modified authorization behaviour: daapd gives out database information without authorization now (as iTunes doesn't send authorization for database entries).
- unicode characters are now handled correctly both in id3 tags and in file names (the latter assuming the os handles them correctly) 
- added a switch that enables verbose output (for debugging purposes).
- added support for uncompressed audio formats and AAC. Metadata contained in these files is not processed yet.
- removed switch for suppressing scans for deleted files, as the scan is one-pass now and scanning for deleted files does not cause a performance hit any more.

0.2

- switched id3 parsing from id3lib to libid3tag (MAD). This significantly speeds up scanning directories.
- fixed a bug that caused daapd to hog the CPU 10 seconds after the directory scan is complete (only happened on Linux)
- added mp3 frame parsing for accurately calculating the song lengths. 
- added support for jumping around in files (via the http range request).
- added an option that purges deleted songs from the database on client connect. This approximately doubles client connect times on fast networks (on slow networks, the transmission time determines the client connect time)

0.1b

- replaced '-v' option and 'Version' with version autodetection
- small cross-platform fixes (includes)
- memory leak in directory scan fixed
- capability to read in multiple files or directories from the command line added
- caching added
- client connects were extremely slow, because all files were unintionally re-scanned. fixed. 
- libhttpd honors 'connection: close' now. this fixes "rebuffering stream" hangs 

0.1a

- added a missing genre validity check that caused segmentation violations
- increased the read buffer size in libhttpd; iTunes 4.0.0 works again 
- fixed a stupid bug in the http response version string; Authentication now works in 4.0 and 4.0.1 
- added a '-v' option

0.1	

(initial release)


ACKNOWLEDGEMENTS

The author wishes to thank Johannes Zander for extensive C++ advice and Todd Larason for help with deciphering the DAA protocol. Andy MacFarland added support for uncompressed files. Ben Whaley provided a revised directory scan that works on Solaris. Eric Shelton reduced memory copies and suggested recompression of file formats iTunes doesn't know about. Alexander Wittig contributed code for handling playlists. 

Several people helped discovering bugs, including:  Lee Lindquist, Ronald Rael Harvest, Lok <lok@meligent.com>, Eagle <eagle@linuxcrypt.com>, Steven Klass, Matt Whiteley, David Buttrick and Alex Southgate.


CONTACT

You can contact the author at daap@deleet.de.


LICENSE

daapd, a server for the DAA protocol
(c) deleet 2003, Alexander Oberdoerster

daapd is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

daapd is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with daapd; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
