Installation Instructions

*************************

Basic Installation

==================

   The `configure' shell script attempts to guess correct values for

various system-dependent variables used during compilation.  It uses

those values to create a `Makefile' in each directory of the package.

It may also create one or more `.h' files containing system-dependent

definitions.  Finally, it creates a shell script `config.status' that

you can run in the future to recreate the current configuration, and a

file `config.log' containing compiler output (useful mainly for

debugging `configure').

   It can also use an optional file (typically called `config.cache'

and enabled with `--cache-file=config.cache' or simply `-C') that saves

disabled by default to prevent problems with accidental use of stale

   If you need to do unusual things to compile the package, please try

to figure out how `configure' could check whether to do them, and mail

diffs or instructions to the address given in the `README' so they can

be considered for the next release.  If you are using the cache, and at

some point `config.cache' contains results you don't want to keep, you

may remove or edit it.

   The file `configure.ac' (or `configure.in') is used to create

  1. `cd' to the directory containing the package's source code and type

  2. Type `make' to compile the package.

  3. Optionally, type `make check' to run any self-tests that come with

  4. Type `make install' to install the programs and any data files and

     source code directory by typing `make clean'.  To also remove the

     files that `configure' created (so you can compile the package for

     a different kind of computer), type `make distclean'.  There is

     also a `make maintainer-clean' target, but that is intended mainly

     for the package's developers.  If you use it, you may have to get

     all sorts of other programs in order to regenerate files that came

     with the distribution.

Compilers and Options

=====================

   You can give `configure' initial values for configuration parameters

by setting variables in the command line or in the environment.  Here

is an example:

   *Note Defining Variables::, for more details.

Compiling For Multiple Architectures

====================================

same time, by placing the object files for each architecture in their

directory where you want the object files and executables to go and run

the `configure' script.  `configure' automatically checks for the

Installation Names

==================

`/usr/local/bin', include files under `/usr/local/include', etc.  You

can specify an installation prefix other than `/usr/local' by giving

   You can specify separate installation prefixes for

architecture-specific files and architecture-independent files.  If you

pass the option `--exec-prefix=PREFIX' to `configure', the package uses

PREFIX as the prefix for installing programs and libraries.

Documentation and other data files still use the regular prefix.

   In addition, if you use an unusual directory layout you can give

options like `--bindir=DIR' to specify different values for particular

kinds of files.  Run `configure --help' for a list of the directories

   If the package supports it, you can cause programs to be installed

with an extra prefix or suffix on their names by giving `configure' the

option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.

`configure', where FEATURE indicates an optional part of the package.

They may also pay attention to `--with-PACKAGE' options, where PACKAGE

is something like `gnu-as' or `x' (for the X Window System).  The

`README' should mention any `--enable-' and `--with-' options that the

package recognizes.

   For packages that use the X Window System, `configure' can usually

find the X include and library files automatically, but if it doesn't,

you can use the `configure' options `--x-includes=DIR' and

`--x-libraries=DIR' to specify their locations.

Specifying the System Type

==========================

`--build=TYPE' option.  TYPE can either be a short name for the system

type, such as `sun4', or a canonical name which has the form:

     CPU-COMPANY-SYSTEM

where SYSTEM can have one of these forms:

   See the file `config.sub' for the possible values of each field.  If

`config.sub' isn't included in this package, then this package doesn't

need to know the machine type.

   If you are _building_ compiler tools for cross-compiling, you should

use the option `--target=TYPE' to select the type of system they will

produce code for.

   If you want to _use_ a cross compiler, that generates code for a

platform different from the build platform, you should specify the

"host" platform (i.e., that on which the generated programs will

eventually be run) with `--host=TYPE'.

Sharing Defaults

================

`configure' looks for `PREFIX/share/config.site' if it exists, then

`PREFIX/etc/config.site' if it exists.  Or, you can set the

`CONFIG_SITE' environment variable to the location of the site script.

A warning: not all `configure' scripts look for a site script.

Defining Variables

==================

environment passed to `configure'.  However, some packages may run

configure again during the build, and the customized values of these

variables may be lost.  In order to avoid this problem, you should set

them in the `configure' command line, using `VAR=value'.  For example:

     ./configure CC=/usr/local2/bin/gcc

causes the specified `gcc' to be used as the C compiler (unless it is

`configure' Invocation

======================

`--help'

`-h'

`--version'

`-V'

     Print the version of Autoconf used to generate the `configure'

     script, and exit.

`--cache-file=FILE'

     Enable the cache: use and save the results of the tests in FILE,

     traditionally `config.cache'.  FILE defaults to `/dev/null' to

     disable caching.

`--config-cache'

`-C'

     Alias for `--cache-file=config.cache'.

`--quiet'

`--silent'

`-q'

     Do not print messages saying which checks are being made.  To

     suppress all normal output, redirect it to `/dev/null' (any error

     messages will still be shown).

`--srcdir=DIR'

     Look for the package's source code in directory DIR.  Usually

     `configure' can determine that directory automatically.

`configure' also accepts some other, not widely useful, options.  Run

`configure --help' for more details.

/*

  Copyright 2010 Nathan Phillip Brink, Ethan Zonca, Matthew Orlando

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/* This file contains the code which both processes (renders) jobs as a slave, and the code which distributes frames to slaves after receiving them from the client portion of the codebase. */

#include "common/config.h"

#include "distrenjob.h"

#include "listen.h"

#include "slavefuncs.h"

#include "mysql.h"

#include "common/asprintf.h"

#include "common/execio.h"

#include "common/options.h"

#include "common/protocol.h"

#include "common/request.h"

#include <arpa/inet.h>

#include <confuse.h>

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#include <sys/stat.h>

#include <sys/types.h>

#include <time.h>

#include <unistd.h>

#include <libxml/encoding.h>

#include <libxml/parser.h>

#include <libxml/tree.h>

#include <libxml/xmlmemory.h>

#include <libxml/xmlreader.h>

#include <libxml/xmlwriter.h>

/* ******************* Structs ************************ */

struct general_info

{

  struct distrenjob head;

  distrend_mysql_conn_t conn;

  struct distrend_config *config;

  struct

  {

    /** general_info.xml */

    char *geninfo;

    /**

     * where to store in-progress uploads or things that should

     * otherwise be on the same filesystem as the rest of the datadir

     * so that it may be rename()d.

     */

    char *tmpdir;

  } files;

  int jobs_in_queue;

  unsigned int free_clients;

  unsigned int rendering_clients;

  unsigned int total_finished_jobs;

  unsigned int total_frames_rendered;

  unsigned int highest_jobnum;

  int hibernate;

  time_t timestamp;

  unsigned long total_render_power;

  unsigned long total_priority_pieces;

};

/* *********************************************

   Function Prototypes

   ********************************************* */

/* ************General Functions************* */

int distrend_do();

int distrend_do_config(int argc, char *argv[], struct distrend_config **config, multiio_context_t multiio);

int distrend_config_free(struct distrend_config *config);

int distrend_handle_request(struct distrend_listens *listens, struct distrend_client *client, struct distren_request *req, void *reqdata, struct general_info *geninfo);

/**

 * client request handlers

 */

int distrend_handle_version(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data);

int distrend_handle_file_post_start(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data);

int distrend_handle_file_post(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data);

int distrend_handle_file_post_finish(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data);

/* functions of some generic sort ...ish */

int distrend_handle_successful_upload(struct distrend_client *client, struct distrend_client_file_post *client_file_post);

/* **************XML Functions**************** */

void update_general_info(struct general_info *geninfo);

int import_general_info(struct general_info *general_info);

int update_xml_joblist(struct general_info *geninfo);

/* **************Test Functions**************** */

int interactiveTest(int test, multiio_context_t multiio, struct general_info *general_info);

/* **************** Main ********************* */

int main(int argc, char *argv[])

{

  /* Parse arguments */

  int counter;

  int test = 0; /*< Interactive mode if 1 */

  int tmp;

  struct general_info general_info;

  multiio_context_t multiio;

  enum clientstatus

  {

    CLIENTSTATUS_UNINITIALIZED = 0,

    CLIENTSTATUS_BUSY = 1,

    CLIENTSTATUS_IDLE = 2

  } clientstatus;

  fprintf(stderr, PACKAGE_STRING "\n\

Nathan Phillip Brink <binki@ohnopub.net>\n\

\n");

#ifdef HAVE_LIST_BRAG

  fprintf(stderr, "Using %s\n", list_brag);

#endif

  clientstatus = CLIENTSTATUS_UNINITIALIZED;

  // xmlinit();

  for(counter = 0; counter < argc; counter ++)

    {

      if(strcmp(argv[counter], "-h") == 0)

      {

    	  fprintf(stderr, "Usage: distrend [option] \nStarts the distrend server\n\t-h\tshow this help\n\t-t\tlaunches queue testing interface \n");

	  return 2;

      }

      else if(strcmp(argv[counter], "-t") == 0)

      {

    	  fprintf(stderr, "Entering into test mode...\n\n");

    	  test = 1;

      }

    }

  multiio = multiio_context_new();

  if(distrend_do_config(argc, argv, &general_info.config, multiio))

    return 1;

  /** preset paths */

  _distren_asprintf(&general_info.files.geninfo, "%s/general_info.xml",

		    general_info.config->datadir);

  _distren_asprintf(&general_info.files.tmpdir, "%s/tmp",

		    general_info.config->datadir);

  distren_mkdir_recurse(general_info.files.tmpdir);

  /** MySQL Connection */

  fprintf(stderr,"Connecting to mysql...\n");

  if(mysqlConnect(&general_info.conn,

		  general_info.config->mysql_user,

		  general_info.config->mysql_host,

		  general_info.config->mysql_pass,

		  general_info.config->mysql_database) )

    {

      fprintf(stderr, "%s:%d: mysqlConnect() failed\n", __FILE__, __LINE__);

      fprintf(stderr, "don't test mysql stuff\n");

      interactiveTest(test, multiio, &general_info);

      return 1;

    }

  fprintf(stderr,"Finished connecting!\n");

  /** Execute test function */

  interactiveTest(test, multiio, &general_info);

  general_info.config->listens = distrend_listens_new(multiio, &general_info, general_info.config->options);

  if(!general_info.config->listens)

    {

      fprintf(stderr, "error initializing listens\n");

      return 1;

    }

  remoteio_generic_data_set(general_info.config->options->remoteio, general_info.config->listens);

  for(counter = 0; general_info.config->listen_ports[counter]; counter ++)

    {

      tmp = distrend_listen_add(general_info.config->listens, general_info.config->listen_ports[counter]);

      if(tmp)

	{

	  fprintf(stderr, "Error listening on port %d\n", general_info.config->listen_ports[counter]);

	  return 1;

	}

    }

  distrend_listen_handler_add(general_info.config->listens, DISTREN_REQUEST_VERSION, &distrend_handle_version);

  distrend_listen_handler_add(general_info.config->listens, DISTREN_REQUEST_FILE_POST_START, &distrend_handle_file_post_start);

  distrend_listen_handler_add(general_info.config->listens, DISTREN_REQUEST_FILE_POST, &distrend_handle_file_post);

  distrend_listen_handler_add(general_info.config->listens, DISTREN_REQUEST_FILE_POST_FINISH, &distrend_handle_file_post_finish);

  /* Main Loop */

  general_info.config->die = 0;

  while(!general_info.config->die)

    {

      multiio_poll(multiio, 15000);

      tabletennis_serve(general_info.config->listens->tabletennis);

      /* Run the watchdog, @TODO: like every 10 mins or something */

      frame_watchdog(general_info.conn);

    }

  distrend_listen_free(general_info.config->listens);

  distrend_config_free(general_info.config);

  xmlcleanup();

  /** free() paths */

  free(general_info.files.geninfo);

  mysqlDisconnect(general_info.conn);

  return 0;

}

/* ********************** Functions ************************* */

int distrend_handle_version(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data)

{

  char *tmp_str;

  struct distren_request_version version;

  if(distren_request_parse_version(req, req_data, &version))

    {

      distrend_send_disconnect(client, "Invalid DISTREN_REQUEST_VERSION packet.");

      return 1;

    }

  if(client->state != DISTREND_CLIENT_PREVERSION)

    {

      distrend_send_disconnect(client, "You have already sent the VERSION command.");

      return 1;

    }

  if(!strncmp(PACKAGE_STRING, version.package_string, DISTREN_REQUEST_VERSION_PACKAGE_STRING_LEN))

    {

      /**

       * The client and I claim to be of the same version of distren :-D

       * Now we will mark the client as valid.

       *

       * We won't increment his time to live, though, because it shouldn't take

       * him that long to auth.

       */

      client->state = DISTREND_CLIENT_PREAUTH;

    }

  else

    {

      /**

       * The client claims to be of a different version of distren.

       * Now we will just send a disconnect packet and disconnect the client.

       */

      _distren_asprintf(&tmp_str, "You have tried to connect to a %s server when your client claims to be running %s. Bye ;-)\n", PACKAGE_STRING, version.package_string);

      if(tmp_str)

	{

	  distrend_send_disconnect(client, tmp_str);

	  free(tmp_str);

	}

      else

	distrend_send_disconnect(client, "Invalid PACKAGE_VERSION :-|.");

      return 1;

    }

  return 0;

}

/**

 * Traversal helper for distrend_client_find_post().

 */

int distrend_client_find_post_traverse(uint32_t *post_id, struct distrend_client_file_post *client_file_post)

{

  if(*post_id == client_file_post->post_id)

    return FALSE;

  return TRUE;

}

/**

 * Find the record for an in-progress client's file posting.

 */

struct distrend_client_file_post *distrend_client_find_post(struct distrend_client *client, uint32_t post_id)

{

  if(list_traverse(client->file_post_list, &post_id, (list_traverse_func_t)&distrend_client_find_post_traverse, LIST_ALTR | LIST_FORW | LIST_FRNT) != LIST_EXTENT)

    return list_curr(client->file_post_list);

  return NULL;

}

/**

 * Finds a post_context based on the post_id and client.

 *

 * Compatible the distren_request_parse_file_post_find_context_func_t.

 */

static distren_request_file_post_context_t distrend_client_find_file_post_context(uint32_t post_id, void *client)

{

  struct distrend_client_file_post *client_file_post;

  client_file_post = distrend_client_find_post(client, post_id);

  if(client_file_post)

    return client_file_post->post_context;

  return NULL;

}

/**

 * Clean up and free a client_file_post

 *

 * Whenever calling this functino, you almost _always_ have to call

 * list_remove_element(client->file_post_list, client_file_post);

 * first.

 */

void distrend_client_file_post_free(struct distrend_client_file_post *client_file_post)

{

  fclose(client_file_post->fd);

  free(client_file_post->filename);

  unlink(client_file_post->file_save_path);

  free(client_file_post->file_save_path);

  free(client_file_post);

}

int distrend_handle_file_post_start(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data)

{

  struct distrend_client_file_post *client_file_post;

  distren_request_file_post_context_t post_context;

  uint32_t post_id;

  char *filename;

  char *str_tmp;

  int ret;

  /**

   * @todo access check!

   */

  fprintf(stderr, __FILE__ ":%d:distrend_handle_file_post_start(): You need to check if a client is actually allowed to upload files somehow!\n", __LINE__);

  /*

   * other servers should be excluded from this check, but we don't

   * store the servertype in client yet.

   */

  if(list_size(client->file_post_list) > 1)

    {

      distrend_send_disconnect(client, "You are trying to upload too many files at once!");

      return 1;

    }

  ret = distren_request_parse_file_post_start(req, req_data, &post_context, &post_id, &filename);

  if(ret)

    {

      distrend_send_disconnect(client, "You sent me an invalid DISTREN_REQUEST_FILE_POST_START packet");

      return 1;

    }

  if(distrend_client_find_post(client, post_id))

    {

      _distren_asprintf(&str_tmp, "Err accepting file: You are trying to upload using post_id=%d while you have already started another upload using the same post_id", post_id);

      distrend_send_disconnect(client, str_tmp);

      free(str_tmp);

      distren_request_file_post_context_free(post_context);

      return 1;

    }

  client_file_post = malloc(sizeof(struct distrend_client_file_post));

  if(!client_file_post)

    {

      distrend_send_disconnect(client, "Error accepting file: out of memory");

      distren_request_file_post_context_free(post_context);

      return 1;

    }

  client_file_post->post_context = post_context;

  client_file_post->post_id = post_id;

  client_file_post->filename = filename;

  _distren_asprintf(&client_file_post->file_save_path, "%s/conn-%d_file_post-%d",

		   geninfo->files.tmpdir, client->connection_id, post_id);

  client_file_post->fd = fopen(client_file_post->file_save_path, "w");

  if(!client_file_post->fd)

    {

      perror("fopen");

      fprintf(stderr, "error: Unable to open ``%s''. See above ``fopen'' error for more details.\n", client_file_post->file_save_path);

      distrend_send_disconnect(client, "Error accepting file: unable to store the file n disk");

      distren_request_file_post_context_free(post_context);

      list_remove_element(client->file_post_list, client_file_post);

      distrend_client_file_post_free(client_file_post);

      return 1;

    }

  list_insert_after(client->file_post_list, client_file_post, 0);

  return 0;

}

int distrend_handle_file_post(struct general_info *geninfo, struct distrend_client *client, struct distren_request *req, void *req_data)

{

  struct distrend_client_file_post *client_file_post;

  void *file_data;

  size_t file_data_len;

  uint32_t post_id;

  char *tmp_str;

  size_t written_len;

  int ret;

  ret = distren_request_parse_file_post(req, req_data, &post_id,

					&distrend_client_find_file_post_context, client,

					&file_data, &file_data_len);

  if(ret)

    {

      distrend_send_disconnect(client, "You sent me an invalid DISTREN_REQUEST_FILE_POST packet");

      return 1;

    }

  client_file_post = distrend_client_find_post(client, post_id);

  if(!client_file_post)

    {