by Tim Parker

IN THIS CHAPTER

• Gopher and Linux

• Configuring Gopher

• Setting Up Your Gopher Directories

• Starting Gopher

• Letting the World Know

Gopher is one of the most useful Internet services available, because it is widely used by beginners and veterans alike. Gopher is a text-based file location system that leads you through a series of hierarchical menus to find specific files you want. Setting up a Gopher site is really just a matter of configuring the Gopher server software and creating a number of logical directory structures with files indexed in a Gopher format.

Gopher works by having a client program (started by a user) connect to a Gopher server and retrieve information about files available on the Internet (or local area network, if the Gopher server is limited to that area). At the end of 1995, there were more than 6,000 Gopher servers on the Internet, all accessible by anyone with a Gopher client. Those servers contain information about more than 10 million items, ranging from text files to movies, sounds, images, and many types of application binaries. Gopher enables you to display and manipulate lists of files, looking for items of interest to you.

If you or the users of your Linux system want to connect to another Gopher server, you need a Gopher client. There are several clients available with Linux distributions, on FTP and BBS sites, and through several other sources. If you don't want to allow users (or yourself) to start a Gopher client, you can use telnet to connect to sites known as public Gopher clients. These sites allow you to log in as an anonymous user and access the Gopher system. Most Gopher client packages offer more than just Gopher programs. Typical Gopher clients enable you to access WAIS indexes, use FTP, and to some extent, interwork with the World Wide Web.

This chapter looks at how you can set up a Gopher server, allowing others to access your machine's Gopher listings. Although we won't go into detail about how you should structure your Gopher service, you will see how to configure your software.

There are currently two versions of Gopher available for Linux systems: Gopher and Gopher+ (Gopher Plus). Gopher is freely available, but Gopher+ is a commercial product. The difference between the two is functionality. If the additional capabilities of Gopher+ are important to you and your Gopher site, you may want to consider purchasing the product. Essentially, Gopher+ adds the following features:

• Makes extended file information available

• Offers a description of a file

• Can retrieve multiple versions of a file at one time (such as ASCII and PostScript simultaneously)

• Allows file retrieval based on search criteria determined by the user

Gopher+ works with Gopher, but Gopher cannot use the advanced features of Gopher+. Gopher+ and Gopher both work with WWW browsers. Gopher+ licenses tend to cost about $100 or $500, depending on the site's conditions.

The versions of Gopher usually offered with Linux come from one of two sources: University of Minnesota Gopher and Gopher+, or GN Public License Gopher. The most recent public version of UM Gopher is version 1.3 (version 2.13 is free only to educational institutions), but the university is no longer working on the freeware Gopher product, instead concentrating on the commercial Gopher+ product. The GN Public License Gopher includes a WWW service but does not provide full functionality at present.

Gopher uses a TCP/IP family protocol known, surprisingly enough, as the Gopher protocol. This is a fairly simple request-answer protocol that is implemented for speed. When Gopher transfers information about a file it knows about (called a Gopher menu file), it follows a set format. This is the format used by Gopher:

<type><display_name><selector string><hostname><port>

The fields in the Gopher menu file have the following meanings:

• type--A one character description of the item (see the following bulleted list for valid codes).

• display_name--The menu or display name, followed by a tab character.
• n selector_string--A unique identifier for a document on each server (usually based on the filename). The selector string is followed by a tab character.

• hostname--The host where the file resides, followed by a tab character.

• port--The port to access the host, followed by a carriage return/line feed pair (usually port 70).

The Gopher+ version of the system adds a few new attributes to each line, including the name of the system administrator responsible for the service, a simple description of the document type (text, for example), the language the file is written in, the last date the file was updated, and the size in bytes.

When a user wants to retrieve a file through the Gopher system, the hostname and port are used to create the connection to the remote server, while the selector string can be used to identify the file to be downloaded.

There are several types of files supported by Gopher, all given a unique one-character type code. The following is a list of valid codes:

• 0 Plain text file

• 
  Directory

• 
  CSO phonebook server (the hostname is the machine to connect to, and the selector string is blank)

• 
  Error

• 
  BinHex Macintosh file

• 
  Binary DOS archive file

• 
  UNIX uuencoded file

• 
  Index-search server

• 
  Pointer to text-based telnet session (the hostname is the machine to connect to, and the selector string is the name to log in as)

• 
  9Binary file

• 
  GIF file

• 
  HTML document

• 
  Graphic image

• 
  Unselectable inline text

• 
  MIME-encapsulated mail document

• 
  Adobe PDF file

• 
  Sound

• 
  Pointer to 3270 telnet session (the hostname is the machine to connect to, and the selector string is the login name)

The Gopher system uses a number of other files on a Linux system, all of which must exist. The files necessary for Gopher are as listed here:

• tn3270 or similar 3270 emulator--Used for telnet 3270 connections.

• kermit or zmodem communications programs--Used for downloading files. The binaries are usually called kermit, sz, sb, and sx.

• graphics utility--If you allow the display of graphics, you need a graphics utility such as xv.

You can modify these requirements if you have a private site (such as for your local area network), but if you are offering open access you should have all the components.

Installing and configuring Gopher (and Gopher+) is a matter of setting a number of configuration options prior to compiling the system software (it is usually not precompiled for you) and configuring some standard files. Gopher+ is done in the same manner, although with some extra parameters. Since Gopher is more likely to be on a Linux site than Gopher+, we'll concentrate on that.

Throughout this section, we will use filenames without full path extensions because it really doesn't matter where you install the Gopher software, as long as the directory paths are set correctly. There is no real standard configuration for directory locations, so feel free to choose whatever works best for you.

The configuration parameters for Gopher (and Gopher+) are in a file called gopherd.conf, which is read by the Gopher daemon gopherd. The default settings generally need a little modification, although many changes are simply changing commented lines to uncommented, and vice versa.

The first step is to create an alias for the Gopher service on your machine. Perform this step with the line that defines the hostalias. There should be a line in the file that looks like this:

hostalias: tpci

The alias is used to find the Gopher server on your system and should not be directly tied to a real machine, so you can make changes whenever you wish. The best approach is to create an alias and tie it to a physical machine with DNS. If you are running a stand-alone machine, you can either use an alias tied to your machine name or use your machine name directly.

You can also control the number of Gopher connections allowed at one time. This is sometimes necessary to prevent a system from bogging down due to excessive user load. The maximum number of connections Gopher allows is given in a file, usually in the directory PIDS_Directory. A line in the gopherd.conf file usually has this variable commented out because early versions didn't implement it properly, or it was unstable. If you want to allow this feature, remove the comment symbol and make sure the directory it points to has the necessary files for your version of Gopher. The line usually looks like this:

#PIDS_Directory: /pids

A better way to handle the load on your system is to use the MaxConnections keyword, which sets the number of clients you support concurrently. You have to experiment to determine the best balance between system load and user service. A good starting point for a fast Linux system (80486 or Pentium CPU) is 15 to 25 users. This variable is set like this:

MaxConnections: 15

If the number of users is exceeded, an error message is generated when the connection is attempted. You can set a number of file decoders for your system. This is used when a user requests a file from Gopher and adds an extension (such as .Z, .gz, or .zip) for a compression or archive utility. The decoders recognize the extension the user supplies and invoke the proper utility to send the file properly decoded. Most gopherd.conf files have the following decoder lines already in the setup:

decoder: .Z /usr/ucb/zcat

decoder: .gz /usr/gnu/bin/zcat

#decoder: .adpcm /usr/openwin/bin/adpcm_dec

#decoder: .z /usr/gnu/bin/zcat

The last two decoders are commented out and can be uncommented if you want to offer files in these formats through Gopher. You can also add other extensions by adding new lines with the binary name (and its full path).

In addition, the amount of time a cache file stays valid should be set. This is controlled by the line using the keyword Cachetime. Set this to a reasonable value, such as 180 seconds. You should have a line that looks like this in the gopherd.conf file:

Cachetime: 180

You can use the gopherd.conf file to restrict access to some files on your system by using the ignore keyword. Usually the gopherd.conf file will have a number of defined ignores, such as these:

ignore: lib

ignore: bin

ignore: etc

ignore: dev

Any file with this type of extension is ignored. If you have a particular file extension you want to protect, add it to the list. For example, if your accounting system uses the extension .acct, you could have the Gopher clients ignore all these files by adding this line:

ignore: acct

Note that these ignore statements work only with file extensions. To be more broad, you can use wildcards and the keyword ignore_patt (for ignore pattern). For example, the line

ignore_patt: ^usr$

ignores any file with the letters usr at the beginning of the name.

In the file gopherdlocal.conf, you have to make two small changes to identify the system administrator, otherwise your system generates many annoying notes. The lines in the gopherdlocal.conf file look like this by default:

Admin: blank

AdminEmail: blank

If you do not change these entries to actual values, Gopher can generate all kinds of weird error messages. The Admin field usually has the administrator's name and sometimes a telephone number. For example, the file could be filled out as follows:

Two files need modification for the compilation process to proceed properly: Makefile.config and conf.h. With many versions of Gopher available on Linux systems, the configuration parameters these files need have already been set, but you should check the values carefully to prevent problems.

The Makefile.config file (used by Makefile to build the executables) is a lengthy file, so you should be careful while moving through it to avoid accidental changes. The important areas to examine are the directory definitions and server and client settings. These are dealt with individually in the following sections.

One setting you may wish to alter is the debugging utility, which is enabled by default in most systems. This can help you get the system running properly, but when the operation is correct, you should recompile the source with the debugging features removed to make the process faster and smaller, as well as to reduce debug information overhead. To remove debugging features, comment out the DEBUGGING line so it looks like this:

#DEBUGGING = -DDEBUGGING

By default this line is probably not commented out.

The directory definitions are usually in a block with five to seven entries, depending on the number of entries for the man pages. A typical directory definition block looks like this:

PREFIX = /usr/local

CLIENTDIR = $(PREFIX)/bin

CLIENTLIB = $(PREFIX)/lib

SERVERDIR = $(PREFIX)/etc



MAN1DIR = $(PREFIX)/man/man1

MAN5DIR = $(PREFIX)/man/man5



MAN8DIR = $(PREFIX)/man/man8

The primary change to most Makefile.config files will be the PREFIX, which is used to set the basic directory for Gopher. The default value is usually /usr/local, although you can change it to anything you want (such as /usr/gopher). The rest of the variables define subdirectories under the primary Gopher directory and are usually acceptable as they are. Each of the subdirectories can be left as is, or you can change it to suit your needs. You can place all the files in one directory, if you want. The meaning of each variable appears in the following list: CLIENTDIR Gopher client software

CLIENTLIB Client help file (gopher.hlp)

MAN1DIR Man pages for gopher client

MAN8DIR Man pages for gopherd

SERVERDIR Gopher server (gopherd) and configuration file (gopherd.conf)

For a Gopher client to run properly on your system, you must modify the CLIENTOPTS line in the Makefile.config file. The two options for the CLIENTOPTS line to control its behavior are as shown here: -DNOMAIL Prevents remote users from mailing files.

-DAUTOEXITONU Allows the Gopher client to be exited with the u command as well as the q command.

To use either or both or these options, add them to the CLIENTOPS line like this:

CLIENTOPTS = -DNOMAIL -DAUTOEXITONU

Four variables relating to the Gopher server must also be set. These specify the host domain name, the port Gopher should use to listen for connections, the location of the data files, and option flags.

The domain name is set with the DOMAIN variable. It should have a leading period in the name, such as this:

DOMAIN = .tpci.com

You do not need to set this variable if the hostname command returns the fully qualified domain name of the server. In that case, leave the value blank.

The SERVERPORT variable defines the port Gopher uses to wait for services and is usually set for TCP port 70. This line usually looks like this:

SERVERPORT = 70

If you are not allowing general access to your Gopher site by Internet users, you can change this value. However, if you want to allow Internet users (even a very small subset) to gain access, you should leave this as port 70. If you are setting up your Gopher site for a small network only, then choose any port number you want (between 1024 and 9999) and make sure all the Gopher clients use that number, too.

The SERVERDATA variable defines the location of the data your Gopher server offers. Its default setting is usually as follows:

SERVERDATA = /gopher-data

Set the variable to point to the file location you use for your Gopher items.

The SERVEROPTS variable accepts a number of keywords that change the behavior of the Gopher service. A typical entry looks like this:

SERVEROPTS = -DSETPROCTITLE -DCAPFILES # -DBIO -DDL

Any keywords after the pound sign are ignored when Makefile runs, so you can adjust its location to set the options you want if the order of the variables allows such a simple approach. The following lists the meaning of the different keywords allowed in the SERVEROPTS entry: -DADD_DATE_AND_TIME Adds dates and times to titles.

-DBIO Used only with the WAIS versions developed by Don Gilbert (wais8b5).

-DDL Provides support for the dl database utility (requires the dl system in a directory variable called DLPATH and the DLOBJS line uncommented out to show the files getdesc.o and enddesc.o locations).

-DCAPFILES Offers backwards compatibility with the cap directory.

-DLOADRESTRICT Restricts user access based on the number of concurrent users (see the following section).

-DSETPROCTITLE Sets the name displayed by ps command (BSD UNIX-based systems only).

The conf.h file is used during the compilation to set other parameters about the Gopher service. The important settings, at least when setting up a Gopher service, are those that relate to the number of queries and timeout variables. These tend to occur at the end of the conf.h file.

The WAISMAXHITS variable defines the maximum number of hits a query to a WAIS database can offer, usually set to around 40. This variable is defined like this:

#define WAISMAXHITS 40

Note that the pound sign is not a comment symbol because this is written in C. The pound sign is an important part of the processor directive and should be left in place. There is no equal sign in the definition, either.

The MAXLOAD variable is used if the -DLOADRESTRICT keyword was used in the SERVEROPTS variable of Makefile.config. The MAXLOAD defines the maximum load average the Gopher service will respond to requests under (this value can be overridden on the command line). The usual definition is like this:

#define MAXLOAD 10.0

The READTIMEOUT and WRITETIMEOUT variables set the amount of time a service waits for a network read or write operation before timing out. The default settings are usually adequate. These lines look like the following:

#define READTIMEOUT (1*60)



#define WRITETIMEOUT (3*60)

The Gopher clients configuration is straight-forward. Begin by defining the Gopher servers the local machine connects to with the CLIENT1_HOST and CLIENT2_HOST entries. The Gopher client chooses one of the two (if both are defined) when it is started. The entries look like this:

#define CLIENT1_HOST "gopher_serv.tpci.com"



#define CLIENT2_HOST "other_gopher_serv.tpci.com"

The ports to be used to connect to the hosts are defined with these options:

#define CLIENT1_PORT 70

#define CLIENT2_PORT 70

If you have a local service and don't want to use port 70 (to prevent access from the Internet, for example), set the proper port values. If only one Gopher server is used, set the second value to 0.

Define the language the Gopher client will use by choosing one value out of a number of options. The default is American English, set by this command:

#define DEFAULT_LANG "En_US"

Other language defines are commented out below this one. If you want to change the default language, comment the American English setting and uncomment the one you want.

When all the configuration changes have been made, you can invoke the compilation process for the client and server with these commands:

make client

make server

Or you can do both client and server systems at once by using the make command with no argument. The programs and data files must be installed, too, using the following command:

make install

Gopher clients have the ability to use WAIS indexes to search for documents, but the system must be configured to allow this. We looked at WAIS in Chapter 49, "Configuring a WAIS Site," so for the sake of providing WAIS index access to Gopher, we will assume you have installed WAIS properly and have WAIS indexes ready for Gopher.

To provide WAIS services through Gopher, you may have to make a change in the WAIS source code. Examine the WAIS source code for a line that looks like this:

if (gLastAnd) printf("search_word: boolean `and' scored/n:);

This line should be commented out to provide Gopher services, so if it is not, add C comment symbols before and after the line, like this:

/* if (gLastAnd) printf("search_word: boolean `and' scored/n:); */

If the line was already commented out (or didn't exist) then you need not make any change. If you changed the line, though, you have to recompile WAIS by changing into the WAIS top directory and running the makefile (type the command make).

Next, examine the Gopher Makefile.config file and look for the WAISTYPE variable. It should be defined on a line like this:

WAISTYPE = #-DFREEWAIS_0_4

Then, you have to link the Gopher and WAIS services. Suppose your Gopher source directory is /usr/gopher/source and the WAIS source directory is /usr/wais/source. You can link these services by entering the following commands:

cd /usr/gopher/source

ln -s /usr/wais/source/include ./ir

ln -s /usr/wais/source/client/ui .

ln -s /usr/wais/source/bin .

When Gopher is recompiled, it will make the links between Gopher and freeWAIS and allow the two services to interwork.

Gopher directories and files are quite simple to set up, and they follow standard naming conventions for the most part. Before you begin, though, you should know which documents and files are to be provided through Gopher to other users, and you should be able to write a short description of each. (If you don't know the contents of a file, either read it or get the author to summarize the file for you.) To begin, we'll assume you will use only a single directory for all your Gopher documents.

Begin by changing to the top directory you use for your Gopher directories (which you may have to create if you haven't already done so). This directory should not be where the Gopher source and configuration files are located, for convenience. Simply choose a useful name and create the directory. For example, to create the Gopher home directory /usr/gopher/data, you would issue a standard mkdir command:

mkdir /usr/gopher/data

Change into your Gopher directory and copy the files you want to make available into it. When you have done that, you can create a descriptive filename for each file (instead of the more obtuse filenames usually used) up to 80 characters long. For example, if you have a filename called q1.sales you may want to rename it to Company_Sales_1997_Q1 to help users identify the contents a little more easily.

The process for providing better filenames is to first create a .cap directory under your Gopher main directory (such as /usr/gopher/data/.cap). For each file in the main directory, you want to create a file in the .cap directory with the same name, but with a name and number. For example, suppose you have a file called q1.sales in /usr/gopher/data. In /usr/gopher/data/.cap you would create a file with the same name, q1.sales, which has the following contents:

Name=Company Sales for the First Quarter, 1997

Numb=1

The Name entry can have spaces or other special symbols in it, because it is echoed as a complete string. The Numb entry is for the location of the entry on your Gopher menu. For example, suppose you had the preceding entry and two other files, shown by using cat to display their contents:

$ cat q1.sales

Name=Company Sales for the First Quarter, 1997

Numb=1



$ cat q2.sales

Name=Company Sales for the Second Quarter, 1997

Numb=2



$cat q3.sales

Name=Company Sales for the Third Quarter, 1997

Numb=3

When these entries are displayed in a Gopher menu they will look like this:

1. Company Sales for the First Quarter, 1997

2. Company Sales for the Second Quarter, 1997

3. Company Sales for the Third Quarter, 1997

The order of filenames in the .cap directory doesn't matter, but you shouldn't have the same Numb entry more than once.

An alternative to using the .cap directory approach (which allows for easy addition of new files) is to use a single master file for each document you are making available. This file goes in your Gopher top directory and is called .names. Here's the .names file for the same three files just mentioned:

$ cd /usr/gopher/data

$ cat .names

# My Gopher main .names file



Path=./q1.sales

Name=Company Sales for the First Quarter, 1997

Numb=1



Path=./q2.sales

Name=Company Sales for the Second Quarter, 1997

Numb=2



Path=./q3.sales

Name=Company Sales for the Third Quarter, 1997



Numb=3

As you can see, this format contains the same information but adds the filename (which was not needed in .cap since the filenames were the same). One advantage to using a .names file is that you can reorder your menu entries much more easily because you only have one file to work with instead of several. Also, the .names file enables you to add an abstract describing the file. For example, you could have the following entry in a .names file:

Path=./gopher

Name=How to Set up A Gopher Service

Numb=16

Abstract=This document shows the steps you need to take to 

set up a Gopher service.

You can get a little fancier with Gopher and have a menu item lead to another menu, or to another machine entirely. This is done with links, controlled by a link file, which ends with .link. A .link file has five pieces of information in it, in the same format as this example:

Name=More Sales Info

Type=1

Port=70

Path=/usr/gopher/data/more_sales

Host=wizard.tpci.com

The Name entry is what a user sees on the Gopher menu, and can be any type of description you want, regardless of what else is in the link file. The Type field has a number showing the type of document the file links to. The following are all valid numbers:

0 Text

1 Directory

2 CSO name server

7 Full text index

8 telnet session

9 Binary

h HTML file

I Image file

M MIME file



s Sound file

These types are the same as the list shown earlier in this chapter for the types of files Gopher supports, although it's a little shorter here.

The Port field is the port for a connection to a remote system (if that's where the link leads), and the Path field is where the file is on the local or remote server. The Host field, not surprisingly, is the name of the host the file resides on. If you are setting up a link to another machine via FTP or WAIS, you need to specify the path to include the service name and any arguments. For example, if your Gopher menu leads users to a file on another machine through FTP, your link file may look like this:

Name=More Sales Info

Type=1

Port=+

Path=ftp:chatton.bigcat.com@/usr/gopher/cats

Host=+

The plus signs used in the Port and Host fields instruct the FTP service on the remote machine to return results to this machine using default ports (such as TCP port 21 for FTP). For a link to a WAIS directory, this would be the format:

Name=More Sales Info

Type=7

Port=+

Path=waisrc:/usr/wais/data

Host=+

Finally, you may want to have a menu item execute a program. You can do this by having the Path field use the exec command:

Path=exec: "args" : do_this

In this command, do_this is the program you want to execute and args are any arguments to be passed to do_this. If you have no arguments to pass, leave the quotation marks empty. This format is a little awkward, but it does work.

A Gopher server can be started either from the rc startup files, from the command line, or from the inetd daemon. From the command line or the rc files, you need a command line similar to this:

/usr/local/etc/gopherd /usr/gopher/gopher-data 70

which starts the daemon with the directory the startup Gopher menus reside in and the port number for connections.

The gopherd command line accepts a number of optional flags to control its behavior, although most mirror entries in the configuration files. Valid flags are listed here: -C Disables directory caching

-c Runs without chroot restrictions

-D Enables debugging

-I inetd is used to invoke gopherd

-L Followed by a value for the maximum load average

-l Log file to record connections (filename follows the option)

-o Specifies an alternate configuration file from gopherd.conf (filename follows the option)

-u Sets the name of the owner running gopherd (valid username must follow the option)

To help secure your system, use chroot to create a separate file system structure for the gopher area (as you did with FTP: see Chapter 48, "FTP and Anonymous FTP Sites"). The -c option is not as secure as running gopherd with chroot active. Also, the -u option should be used to make gopherd run as a standard user's process, instead of as root. This helps protect against holes in the daemon that a hacker could exploit.

If you want to run Gopher under inetd (started whenever a request for the service arrives), modify the /etc/services and /etc/inetd.conf file to include a line for Gopher. Normally, the entry in /etc/services looks like this:

gopher     70/tcp     

And the entry in /etc/inetd.conf looks like

gopher     stream     tcp     nowait     root 



    /usr/local/etc/gopherd gopherd -I -u username

where username is the name of the user to run gopherd as (you can set up a specific account for gopher in /etc/passwd with standard permissions).

Once the Gopher server process is up and running, you can test your Gopher installation. You will need a Gopher client. Use the Gopher client to connect to your Gopher server (using your host name), and you should see the top directory of your Gopher resources. Another way to test your Gopher system is to use telnet. Use Telnet to connect to the Gopher port, using a command like this:

telnet gopher 70

If the connection is properly made, you will see your Gopher system on the screen.

Yet another alternative to test your system is to use the program gopherls, which requires the name of the directory your Gopher source resides in. To start gopherls, you would issue the command

gopherls /usr/wais/gopher/data

specifying your Gopher data directory. You can use this technique to test new Gopher directories as you develop them.

Because you have spent a lot of time setting up your Gopher service, you can now let everyone else on the Internet know about it. (Of course, you should only do this when your Gopher service is ready, and if you want to allow general access. Don't follow these steps if you are granting access only to a few people or your local area network.)

To have your Gopher service listed in the main Gopher service directories, send an e-mail message to the following address: