This program emulates a QuickMail directory server. Queries are sent by QuickMail to the qmdsx server via AppleTalk as if it were a "normal" QuickMail directory server. The qmdsx program then takes the query, proerly formats it, and forwards it via the Lightweight Directory Access Protocol (LDAP) to an X.500 directory server. The results are then received back from the program, reformatted, and sent back to the QuickMail client user.
This program was written using BSD style system calls. It has been tested under FreeBSD and SunOS 4.1.3. It requires the NetATALK Appletalk for Unix stack and the LDAP V3.0 Unix client libraries.
This program was written to emulate a QuickMail directory, and there is not a one-to-one mapping between QuickMail and X.500 directory operations. This has lead to the imposition of the following restrictions/caveats:
The most serious consequence of this behavour is that the qmdsx program cannot perform approximate or soundex matching. This is because if the resulting field does not exactly match the original user specification the QuickMail client will reject it even if the qmdsx program finds a match. I really can't imagine why CE Software would put the burden of verifying fields on the client, but they do and we lose.
If you want to provide access to multiple directories you will need to run a seperate instance of the qmdsx program for each directory base you want to search. These will appear as named directories in the "Directory Services..." submenu. The user selects the directory to search before pressing the address button; thereafter the "Find" button will issue queries for the specified directory.
For example, assume you are looking up my record. You would supply first name "michael" and last name "newell". The QuickMail client sends "mich" and "newe" respecively. Qmdsx then converts this to query "(cn=mich* newe*)" and forwards the query. Qmdsx scans the results looking for a common name field which starts with "mich"; if it finds one it takes the first n non-blank characters and returns them as the first name. Thus fiield "michael c. newell" would yeild first name "michael".
This can be a problem with some names. For people that don't like using their first names but like using their first initial the lookup will fail to find a matching name. Thus if someone is listed only as "J Robert Smith" and the user supplies "robert" as the first name the qmds program will not find this match. There are some possible approaches to solving this in most cases, so this not be a problem in future releases.
The following bugs have been found during testing. They have not been fixed...
The user interface is the same as all other QuickMail directories. The directory server is selected via the "Directory Services..." form (the precise method of invoking this form varies between various versions of QuickMail). The available directories appear just like their QuickMail counterparts. The user can double click on the desired directory to select it.
Once a directory is selected, the user can press the "Find" on the address form to pop up the directory search form. The user then may fill in a first and last name to be located. The user should not fill in a department; department searching is not supported in this release. Once the first and/or last name have been supplied the user may press the "OK" button to launch the search.
While the search is in progress the cursor will change to a spinning disc. When the search completes the cursor will change back to an arrow and the desired name(s) may be selected by double-clicking them. If the disc stops spinning that means the directory query has timed out and no (more) reponses will be displayed. If this happens the machine is not locked up - the user may proceed as if the search completed normally. Single clicking (highlighting) a name will cause the parametric information about the entry to be displayed in the lower part of the selection popup. This is useful for selecting among a number of similar names.
Once the entries have been selected QuickMail will install them in the address list. The correct data may be verified by double-clicking the entry. When the user is satisfied with the address list he/she/it may press the "OK" button to close the address composition popup.
The program runs on Unix hosts and is invoked from a standard Unix-type command line. The form of the command is:
qmdsx [options] searchbase LDAP-server [LDAP-Port]
where the parameters are:
In addition the program accepts a number of options. Specifying a command line with an invalid optioin causes them to be listed with short descriptions. The options consist of keyords preceeded by a dash and, in some cases followed by a parametric value. The options are:
- searchbase
- The directory search base to use. The qmdsx program will perform subtree searches from this level. Thus specifying
"ou=Headquarters,o=National Aeronautics and Space Acministration,c=US" will generate searches in the NASA Headquarters directory. Note that since the argument requires special characters it must be enclosed in quotes or apostrophes.
Default: none. This parameter must be specified. If no search base is provided and error is returned and the program exits.
- LDAP-Server
- Specifies the host name of the LDAP server to be used. The server must be running the Version 3.0 LDAP server code.
Default: the value contained in the environment variable "LDAP_SERVER_NAME". If no value is given and the variable is not set an error is returned and the program exits.
- LDAP-Port
- Specifies the LDAP service port on the LDAP-Server host.
Default: the value contained in the environment variable LDAP_SERVER_PORT or zero, which causes the LDAP library to use its default port.
- -nolog
- Turns off logging. Normally the program logs "interesting" information to the daemon facility of the syslog program. If you don't want to see this information specify "-nolog". Note that only informational messages will be suppressed by this flag; errors and debug messages will continue to be logged.
Default: logging is enabled.
- -nodaemon
- Causes the program to not detach from the terminal. Useful for finding out when core dumps occur.
Default: the program detaches and runs as a daemon.
- -debug
- Turns on tons of debug log messages using syslog facility daemon.debug (daemon.info on SunOS 4.1.3).
Default: debug messages are disabled.
- -wdir dirspec
- Causes the program to perform a chdir to directory "dirspec" when it detaches.
Default: "/tmp".
- -dump
- Causes the program to print a dump of its operational parameters to stderr before it detaches.
Default: operational parameters are not displayed.
- -limit value
- Sets the X.500 query size limit to value responses. Note that the directory may impose a limit smaller than yours.
Default: internally defined constant value. This value is currently set to 30.
- -dirname directory-name
- This is the name will use to name itself to QuickMail users via the "Directory Services..." panel. Each instance of the program must have a unique name (although this is not enforced.)
Default: use the value supplied as the search base, truncated to conform to Appletalk name binding characteristics.
Note: for beta versions the string "(beta) " is always prepended to the directory name. This insures that users know the directory server is beta code.
- -dirzone zone-name
- Specifies the zone in which the directory will appear. The zone must be "visible" on the network on which the directory server is running. Note that if you wish to include spaces or other special characters in this name it must be enclosed in quotes or the special characters must be escaped. Note that a seed router must allow the zone on the network.
Default: "*" which indicates the default zone for the network on which the server is running.
- -gateway gateway-name
- Specifies the name of the QuickMail SMTP gateway mail center. This mail center must be visible to the end user's workstations, but does not have to be visible to the qmdsx directory server. This is returned as the mailcenter value.
Default: the value contained in the environment variable ATK_QMSERVER_NAME, or if the variable is not defined "QMGATE".
- -gwzone gateway-zone-name
- Specifies the name of the zone in which the QuickMail SMTP gateway resides. This zone must be visible to the end user's workstations, but does not have to be visible to the qmdsx directory server. This is returned as the Zone value. Note that if you wish to include spaces or other special characters in this name it must be enclosed in quotes or the special characters must be escaped.
Default: the value contained in the environment variable ATK_QMSERVER_ZONE, or if that variable is not defined "*" which QuickMail interprets as "No Zone".
The program allows some specification defaults to be provided via environment variables. This allows the invoker to avoid having to re-type common information. Note that the values supplied as command line options always override those specified by environment variables. The environment variables are:
- LDAP_SEARCH_TEMPLATE
- Specifies a template for computing the search base from the value supplied on the command line. This was included to make creation of multiple directories within a common structure easier to specify. The template is an sprintf format string containing one "%s"; the location of this "%s" is replaced by the value supplied on the command line. For example, suppose the luser enters
setenv LDAP_SEARCH_TEMPLATE "ou=%s,o=Org Name,c=US" qmds Headquarters qmds "Field Office"The result would be two directories with two search bases - "ou=Headquarters,o=OrgName,c=US" and "ou=Field Office,o=Org Name,c=US".
- LDAP_SERVER_NAME
- Specifies the default value for the LDAP server host machine.
- LDAP_SERVER_PORT
- Specifies the default value for the LDAP server host port.
- ATK_QMSERVER_NAME
- Specifies the default mail center name for the QuickMail SMTP gateway.
- ATK_QMZONE_NAME
- Specifies the default zone name for the QuickMail SMTP gateway.
The program responds to the following signals:
qmdsx -gwname SMTPGW -gwzone "Mail server zone" -dirname "Headquarters" "ou=Headquarters,o=Orgname,c=US" x500.myorg.org qmdsx -gwname SMTPGW -gwzone "Mail server zone" -dirname "Region 1" "ou=Region 1,o=Orgname,c=US" x500.myorg.org
(Note: the quotes aroung "Headquarters" are not strictly necessary since it does not contain any special characters or spaces, but I tend to like to make my command lines consistant for easy debug. You don't have to.)
This can be simplified via the appropriate use of environment variables:
(Note that the setenv command is used for csh and csh-derived shells. Use the set command if you don't have a setenv command.setenv LDAP_SERVER_NAME x500.myorg.org setenv LDAP_SEARCH_TEMPLATE "ou=%s,o=Orgname,c=US" setenv ATK_SERVER_NAME SMTPGW setenv ATK_SERVER_ZONE "Mail server zone"
Given these environment variables, the qmdsx commands become:
qmdsx "Headquarters" qmdsx "Region 1"
This program was based on an earlier program written by Steve Schoch at the NASA Ames Research Center. This program is a total re-write of Steve's code using LDAP and NetAT rather than ISODE and CAP (see below). This makes the code much more portable.
This program also operates totally asynchronously using child processes to handle each request; Steve's original program was a single process. My approach allows the program to operate (in my opinion) more efficiently and also isolates the main program (which implements the directory frontend) from LDAP bugs that may cause a child process to crash.
Finally my design allows easy installation of modules for other purposes, such as interfacing with the Star*Nine directory products and creating servers for other email systems.
The NetAT Appletalk for Unix stack provides the Appletalk interface. This stack is available from "ftp://terminator.rs.itd.umich.edu/unix/netat". This stack was chosen because it is a kernel-native Appletalk implementation which is MUCH faster than the Columbia Appletalk Package (CAP) and allows for unix-like interfacing for I/O calls. The Lightweight Directory Access Protocol (LDAP) package (also from terminator at UMich) was used to implement the X.500 directory interface, thus elimintating the need to port the ISODE or some other UNIX OSI programming environment to the server host.
Ok, enough advertisement...
Field Bytes Purpose unknown 3 Unknown data; always zero function 1 Function to be performed: 1=lookupunknown 4 Unknown data; always zero first 4 First four characters of first name, "*" padded last 4 First four characters of last name, "*" padded depart 4 First four characters of department name, "*" padded
(Note that some fields are unknown; this is because Steve Schoch reverse engineered the protocol using tcpdump and was thus unable to determine what all the fields mean.
A server becomes visible to the QuickMail system by advertising itself on the Appletalk LAN using the Appletalk Name Binding Protocol (NBP). The entry for the server looks like:
server-name:NameServer@zone-namewhere "server-name" is the name by which the server will be known in the server selection panel, and "zone-name" is the zone in which the server will appear. NBP binds this name to a (network,node,socket) tuple. QuickMail sends the ATP request to this tuple, then waits for responses.
The responses are packages which contain some protocol information and single directory entries consisting of variable length fields separated by tabs. The last field is terminated by a carriage return character. The respone packet looks like:
and the fields contained in the data field are
Field Bytes Purpose unknown 3 Unknown data; always zero function 1 Function to be performed: 2=server is down
3=result
4=doneunknown 4 Unknown data; always zero data variable Data to be sent back to the client <cr> 1 terminating carriage return character
The handshake process is as follows:
Field Purpose X.500 Attribute last Last name surname first First name extracted from commonName SMTP Email address rfc822Mailbox gateway Quickmail SMTP gateway name supplied on command line gw Zone Zone for QuickMail SMTP gateway supplied on command line workgroup Workgroup name unused wkg server Workgroup server name unused wkg zone Workgroup zone name unused department Department/office unused address 1 First address postalAddress address 2 Second address homePostalAddress city City part of address unused state State part of address unused zipcode ZIP code unused country Country part of address unused phone 1 First telephone number telephoneNumber phone 2 Second telephone number facsimileTelephoneNumber
QM Client Data Flow Server send initial packet -----> receive packet, initate search receive and display first entry <----- send first entry back request next entry -----> receive packet, get next entry receive and display next entry <----- send next entry back . . . . . . . . . request next entry -----> receive packet, no more entries terminate processing <----- send "done" packet
In order to improve performance and simplify the code, I chose to have child processes created to respond to each request. This means that either (a) the main process must give up the (network,node,socket) typle to the child so that the child can process next-entry requests from QuickMail, or (b) the main process must receive the next-entry requests and forward them to the appropriate child process. I chose (b) because (a) requires that the main process re-register itself through NBP with a new socket number; this gives rise to potiential race conditions if the server becomes heavily used by multiple QuickMail clients.
The following diagram shows the message flows between the QuickMail client, the main process, and the child processes:
When the main process starts up it registers its (network,node,socket) tuple via NBP then waits for QuickMail client transactions to arrive. When a transaction arrives the program extracts the transaction-id field from the request. QuickMail clients always use the same transaction-id for the request stream, so this is used to keep track of which child is servicing which stream.
The children are kept track of by a list of transaction stream blocks. The format of these blocks is:
Field Type Purpose next pointer Pointer to next transaction block pid pid_t Process ID of child handling this transaction fd int File descriptor for write pipe to child tid int Transaction ID number seqnum int Sequence number for request/reply
The main process searches the list of transaction blocks for a child whose tranaction-id matches that of the ATP packet. If such a child is found the packet is written to that child's pipe, and the main process goes back to read the next request from the Appletalk socket. The main process must send the ATP transaction block as well as the QuickMail client's socket address (contained in the ATP transaction block) and the request transaction ID. When the child responds it must use this ID in the response message; sending this ID to the child allows the child to reply directly to the QuickMail client rather than having to send the response through the main process.
If no child process is found the main process creates a new transaction block, fills it with the proper information (e.g. a new pipe fd is created, etc.) and the transaction block is added to the list. The main process then forks the child and goes back to listening on the Appletalk socket. Note that for the initial request the main process does not have to forward the ATP transaction block to the child, since the child is a clone of the parent.
When a child terminates the main process simply removes the transsaction block from its list. If the child terminated before the QuickMail handshake completed the QuickMail client will time out and no action need be taken. QuickMail may re-send the transaction; in which case the main process will see this as a new request and start up a new child. This makes the program self-repairing in cases where resources are temporarily unavailable to procreate.
I wanted the child to be able to start sending back responses as soon as possible, so I chose to call ldap_result() in such a way that it returns each record from the X.500 directory individually rather than as a single block response. This way the user sees the directory entries slowly filling the selection box, rather than seeing a blank box for a long time period. Also the QuickMail client only listens to responses for a fixed period of time (I'm sure that can be fixed...) and if the entire X.500 query takes longer than that period no results can be returned. The implemented technique at least allows the user to receive partial results as soon as they are available.
When the child receives a response from the ldap_result() call it immediately calls the atp_sresp() routine to reply to the request. It then blocks until it is able to read the QuickMail client's request for the next entry from its pipe. It is possible that the QuickMail client will not respond - for example it stops responding if it does not see all the responses during its listen window. To avoid having children hang around forever (and we all know how annoying children are when they are asking for something) a timer is set to abort the child if no response is recieved in a set time period.
Assuming the child receives the "get-next" request from the client, it returns to the "ldap_result() routine and gets the next result. When all results have been received the child sends a response to the QuickMail client tagged with the done tag. The QuickMail client then closes out the transaction and changes the cursor back to an arrow. The child detaches from the LDAP server then performs an "exit(0)" call to terminate. When the child terminates the main process receives the SIGCHLD signal which causes it to delete the child's transaction block.
The current code includes a timeout option on the "ldap_result()" call. This is designed to allow us to try to send "wait" type messages back to the client so it won't time out. So far this hasn't worked, but I'd like to get it to do so.
Unfortunately there are some problems with the mapping of QuickMail to X.500 requests, and reverse mapping the responses. These require special handling, as described below.
When a QuickMail request is received only the first four characters of the request fields are available. Thus if a user requested information on user "smith", the child only sees "smit". This means that the child must prepare and send responses for anything that starts with "smith"; e.g. "Smithson", "Smithers", etc. These non-matching requests are ignored by the QuickMail client, so the only real harm done is that the search may return more responses than allowed; in which case the user will not recieve all valid responses from the directory. Also the more extensive searching may take more time, which may exceed the QuickMail client's listing time period.
In addition, if the user requests a specific first name this needs to be mapped to the X.500 common name format, since the X.500 directory does not have a "firstName" attribute. Thus for the query "(Jim*,Smith,*)" the client sees "(Jim*,Smit,****)" which it maps to X.500 query "(cn=Jim* Smit*)". This query will ONLY succeed if indeed there is a record of this form. If the desired "Jim Smith"'s real name is "R James Smith" (or even "R Jim Smith") and there is no alias "Jim Smith" the query will fail.
The child now needs to figure out what to return for the first name part. The chile does this by scanning all the available common names of the record, looking for one that starts with the same first characters up to the "*" as originally sent by the client. If the child finds such a name it returns the leading non-blank characters as the first name. Thus if the client sent "Jim* Smith" and a record with commonName "Jimmy Bob Smith", first name "Jimmy" and last name "Smith" will be returned. If the wants "Jane Smith", the client sends this as "JANSMIT" and the child converts this to "(cn=Jane* Smit*)". If a record comes back with "(cn=Janey Smithson)" then the child will send back first name "Janey", last name "Smithson".
This can be a real problem because some people are listed in the directory with similar names. For example, "Jane Smith" could easily be listed as both "(cn=Jane Smith)" and "(cn=Janey Smith)". Since the child doesn't know what the user wanted, it returns whichever of the first names it sees first - hence a query for "(jane,smith,*)" gets turned into "JANESMIT" which then may elicit the response "(Janey,Smith)" which would be rejected by QuickMail. I dont' know how to avoid this.