Installation Instructions

*************************

Basic Installation

==================

   The `configure' shell script attempts to guess correct values for

various system-dependent variables used during compilation.  It uses

those values to create a `Makefile' in each directory of the package.

It may also create one or more `.h' files containing system-dependent

definitions.  Finally, it creates a shell script `config.status' that

you can run in the future to recreate the current configuration, and a

file `config.log' containing compiler output (useful mainly for

debugging `configure').

   It can also use an optional file (typically called `config.cache'

and enabled with `--cache-file=config.cache' or simply `-C') that saves

disabled by default to prevent problems with accidental use of stale

   If you need to do unusual things to compile the package, please try

to figure out how `configure' could check whether to do them, and mail

diffs or instructions to the address given in the `README' so they can

be considered for the next release.  If you are using the cache, and at

some point `config.cache' contains results you don't want to keep, you

may remove or edit it.

   The file `configure.ac' (or `configure.in') is used to create

  1. `cd' to the directory containing the package's source code and type

  2. Type `make' to compile the package.

  3. Optionally, type `make check' to run any self-tests that come with

  4. Type `make install' to install the programs and any data files and

     source code directory by typing `make clean'.  To also remove the

     files that `configure' created (so you can compile the package for

     a different kind of computer), type `make distclean'.  There is

     also a `make maintainer-clean' target, but that is intended mainly

     for the package's developers.  If you use it, you may have to get

     all sorts of other programs in order to regenerate files that came

     with the distribution.

Compilers and Options

=====================

   You can give `configure' initial values for configuration parameters

by setting variables in the command line or in the environment.  Here

is an example:

   *Note Defining Variables::, for more details.

Compiling For Multiple Architectures

====================================

same time, by placing the object files for each architecture in their

directory where you want the object files and executables to go and run

the `configure' script.  `configure' automatically checks for the

Installation Names

==================

`/usr/local/bin', include files under `/usr/local/include', etc.  You

can specify an installation prefix other than `/usr/local' by giving

   You can specify separate installation prefixes for

architecture-specific files and architecture-independent files.  If you

pass the option `--exec-prefix=PREFIX' to `configure', the package uses

PREFIX as the prefix for installing programs and libraries.

Documentation and other data files still use the regular prefix.

   In addition, if you use an unusual directory layout you can give

options like `--bindir=DIR' to specify different values for particular

kinds of files.  Run `configure --help' for a list of the directories

   If the package supports it, you can cause programs to be installed

with an extra prefix or suffix on their names by giving `configure' the

option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.

`configure', where FEATURE indicates an optional part of the package.

They may also pay attention to `--with-PACKAGE' options, where PACKAGE

is something like `gnu-as' or `x' (for the X Window System).  The

`README' should mention any `--enable-' and `--with-' options that the

package recognizes.

   For packages that use the X Window System, `configure' can usually

find the X include and library files automatically, but if it doesn't,

you can use the `configure' options `--x-includes=DIR' and

`--x-libraries=DIR' to specify their locations.

Specifying the System Type

==========================

`--build=TYPE' option.  TYPE can either be a short name for the system

type, such as `sun4', or a canonical name which has the form:

     CPU-COMPANY-SYSTEM

where SYSTEM can have one of these forms:

   See the file `config.sub' for the possible values of each field.  If

`config.sub' isn't included in this package, then this package doesn't

need to know the machine type.

   If you are _building_ compiler tools for cross-compiling, you should

use the option `--target=TYPE' to select the type of system they will

produce code for.

   If you want to _use_ a cross compiler, that generates code for a

platform different from the build platform, you should specify the

"host" platform (i.e., that on which the generated programs will

eventually be run) with `--host=TYPE'.

Sharing Defaults

================

`configure' looks for `PREFIX/share/config.site' if it exists, then

`PREFIX/etc/config.site' if it exists.  Or, you can set the

`CONFIG_SITE' environment variable to the location of the site script.

A warning: not all `configure' scripts look for a site script.

Defining Variables

==================

environment passed to `configure'.  However, some packages may run

configure again during the build, and the customized values of these

variables may be lost.  In order to avoid this problem, you should set

them in the `configure' command line, using `VAR=value'.  For example:

     ./configure CC=/usr/local2/bin/gcc

causes the specified `gcc' to be used as the C compiler (unless it is

`configure' Invocation

======================

`--help'

`-h'

`--version'

`-V'

     Print the version of Autoconf used to generate the `configure'

     script, and exit.

`--cache-file=FILE'

     Enable the cache: use and save the results of the tests in FILE,

     traditionally `config.cache'.  FILE defaults to `/dev/null' to

     disable caching.

`--config-cache'

`-C'

     Alias for `--cache-file=config.cache'.

`--quiet'

`--silent'

`-q'

     Do not print messages saying which checks are being made.  To

     suppress all normal output, redirect it to `/dev/null' (any error

     messages will still be shown).

`--srcdir=DIR'

     Look for the package's source code in directory DIR.  Usually

     `configure' can determine that directory automatically.

`configure' also accepts some other, not widely useful, options.  Run

`configure --help' for more details.

/*

  Copyright 2010 Nathan Phillip Brink, Ethan Zonca, Matthew Orlando

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "common/config.h"

#include "distrenjob.h"

#include "slavefuncs.h"

#include "common/asprintf.h"

#include "common/execio.h"

#include "common/protocol.h"

#include "common/remoteio.h"

#include <curl/curl.h>

#include <curl/easy.h>

#include <curl/types.h>

#include <archive.h>

#include <archive_entry.h>

#include <errno.h>

#include <stdio.h>

#include <string.h>

#include <unistd.h>

#include <stdlib.h>

#include <sys/stat.h>

#include <fcntl.h>

#include <time.h>

#define DEBUG 0

/**

   Grabs the xml DOM node reached by an XPath.

   @param path an XPath that lead to DOM node

   @return the first node associated with the path or NULL if there is no match

 */

xmlNodePtr xml_quickxpath(xmlXPathContextPtr xpathctxt, xmlChar *path)

{

  xmlNodePtr toreturn;

  xmlXPathObjectPtr xmlxpathobjptr;

  xmlxpathobjptr = xmlXPathEval(path, xpathctxt);

  if(!xmlxpathobjptr

     || !xmlxpathobjptr->nodesetval->nodeNr)

    {

      fprintf(stderr, "XPath resolution failed for ``%s'' in ``%s'' (``%s'')\n", path, xpathctxt->doc->name, xpathctxt->doc->URL);

      return (xmlNodePtr)NULL;

    }

  toreturn = *(xmlxpathobjptr->nodesetval->nodeTab);

  xmlXPathFreeObject(xmlxpathobjptr);

  return toreturn;

}

/** Deletes job data from the disk. @TODO: add error checking! */

int delete_jobdata(int jobnum, char *datadir)

{

  char *jobpath;

  char *jobcont;

  _distren_asprintf(&jobpath, "%s/%d", datadir, jobnum);

  _distren_asprintf(&jobcont, "%s/%d/*", datadir, jobnum);

  remove(jobcont);

  rmdir(jobpath);

  fprintf(stderr, "Removed files in %s/%d/ if everything was successful\n", jobpath, jobnum);

  free(jobpath);

  free(jobcont);

  return 0;

}

/** Function referenced by curlget() to write data to disk. */

size_t curl_writetodisk(void *ptr, size_t size, size_t nmemb, FILE *stream)

 {

    return fwrite(ptr, size, nmemb, stream);

  }

/** Helper function for cURL's progress display */

int curl_progress(char *Bar, double t, double d, double ultotal, double ulnow)

{

fprintf(stderr,"Downloading: %f%% complete\r",d/t*100);

return 0;

}

/** Retrieves a URL with cURL and saves it to disk */

CURLcode curlget(char *url, char *out){

  fprintf(stderr,"Preparing to download %s\n",url);

  double *Bar; // Stores cURL progress display info

  CURL *curl;

  CURLcode res;

  FILE *outfile;

  res = CURLE_FAILED_INIT;

  curl = curl_easy_init();

  if(curl)

    {

    outfile = fopen(out, "w"); // Open where we're writing to

    if(outfile == NULL)

      return 2; // Output dir doesn't exist

    curl_easy_setopt(curl, CURLOPT_URL, url);

    curl_easy_setopt(curl, CURLOPT_WRITEDATA, outfile);

    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curl_writetodisk); // this MUST be set for win32 compat.

    curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);

    curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, (curl_progress_callback)&curl_progress);

    curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, &Bar);

    res = curl_easy_perform(curl);

    curl_easy_cleanup(curl);

    fclose(outfile);

  }

  fprintf(stderr,"\n"); // Clears out he progressbar's carriage return

  return res; // 0 is OK, 1 is 404 or other error

}

/** Posts a file to a url with cUrl */

CURLcode curlpost(char *filename, char *url, int jobnum, int framenum, int slavekey)

{

  char *targetname = "uploadedfile"; // Name of the target in the php file on the server (Don't change me unless you have different PHP code)

  CURL *curl;

  CURLcode res;

  struct curl_httppost *formpost=NULL;

  struct curl_httppost *lastptr=NULL;

  struct curl_slist *headerlist=NULL;

  static const char buf[] = "Expect:";

  char *sjobnum;

  char *sframenum;

  char *sslavekey;

  if(DEBUG)

    fprintf(stderr,"Uploading to %s\n", url);

  _distren_asprintf(&sjobnum,"%d",jobnum);

  _distren_asprintf(&sframenum,"%d",framenum);

  _distren_asprintf(&sslavekey,"%d",slavekey);

  curl_global_init(CURL_GLOBAL_ALL);

  /* jobnum field... */

   curl_formadd(&formpost,

                &lastptr,

                CURLFORM_COPYNAME, "jobnum",

                CURLFORM_COPYCONTENTS, sjobnum, CURLFORM_END);

  /* framenum field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "framenum",

               CURLFORM_COPYCONTENTS, sframenum, CURLFORM_END);

  /* slavekey field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "slavekey",

               CURLFORM_COPYCONTENTS, sslavekey, CURLFORM_END);

  /* upload field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, targetname,

               CURLFORM_FILE, filename,

               CURLFORM_END);

  /* filename field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "filename",

               CURLFORM_COPYCONTENTS, filename,

               CURLFORM_END);

  /* submit field, not usually needed, just in case... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "submit",

               CURLFORM_COPYCONTENTS, "send",

               CURLFORM_END);

  res = CURLE_FAILED_INIT;

  curl = curl_easy_init();

  headerlist = curl_slist_append(headerlist, buf);

  if(curl)

    {

    /* Setting the URL to get the post, and the contents of the post */

    curl_easy_setopt(curl, CURLOPT_URL, url);

    curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);

    res = curl_easy_perform(curl);

    curl_easy_cleanup(curl);

    /* cleanup the formpost junk */

    curl_formfree(formpost);

    curl_slist_free_all (headerlist);

    free(sjobnum);

    free(sframenum);

    free(sslavekey);

    }

  return res;

}

/** Replaces wordtoreplace with replacewith in conffile (relative to SYSCONFDIR) */

int conf_replace(char *conffile, char *wordtoreplace, char *replacewith){

  int maxlinelen = 120;

  char *fileOrig;

  char *fileRepl;

  _distren_asprintf(&fileOrig, "%s/%s", SYSCONFDIR, conffile);

  _distren_asprintf(&fileRepl, "%s%s.edited", SYSCONFDIR, conffile);

  char buffer[maxlinelen+2];

  char *buff_ptr, *find_ptr;

  FILE *fp1, *fp2;

  size_t find_len = strlen(wordtoreplace);

  fp1 = fopen(fileOrig,"r");

  fp2 = fopen(fileRepl,"w");

  if (fp1 ==NULL){

    fprintf(stderr, "%s doesn't exist\n",fileOrig);

    return 0;

  }

  else if(fp2 ==NULL){

    fprintf(stderr, "Can't write a file to disk! Check permissions.\n");

    return 0;

  }

  else{

    while(fgets(buffer,maxlinelen+2,fp1))

      {

        buff_ptr = buffer;

        while ((find_ptr = strstr(buff_ptr,wordtoreplace)))

        {

           while(buff_ptr < find_ptr)

             fputc((int)*buff_ptr++,fp2);

           fputs(replacewith,fp2);

           buff_ptr += find_len;

         }

         fputs(buff_ptr,fp2);

      }

    rename(fileRepl, fileOrig);

  }

  fclose(fp2);

  fclose(fp1);

  fprintf(stderr,"Wrote conf file...\n");

return 1; // Success

}

/* Executors */

/** Executor function for Blender operations */

int exec_blender(char *input, char *output, int frame)

{

  int ret;

  char *frame_str;

  _distren_asprintf(&frame_str, "%i", frame);

  char *command = "blender"; // @TODO: We currently expect this to be in PATH

  char *cmd[] = { command, "-b", input, "-o", output, "-f", frame_str, "-t", "0", (char *)NULL }; // arguments for blender

  if(DEBUG)

    fprintf(stderr,"Preparing to execute command: %s -b %s -o %s -f %s\n", command, input, output, frame_str);

  char buf[20];

  struct execio *testrem;

  size_t readlen;

  if(DEBUG)

    fprintf(stderr,"Executing: %s\n", frame_str);

  ret = execio_open(&testrem, command, cmd);

  if(ret)

    {

      fprintf(stderr, "Error running blender\n");

      return 1;

    }

  buf[19] = '\0';

  while(!execio_read(testrem, buf, 19, &readlen))

    {

       buf[readlen] = '\0';

       if(DEBUG)

         fprintf(stderr, "read \"%s\"\n", buf);

    }

  execio_close(testrem);

  free(frame_str);

  return ret;

}

void xmlinit()

{

  xmlInitParser();

  xmlXPathInit();

}

void xmlcleanup()

{

  xmlCleanupParser();

}

/** Creates directories recursively */

int distren_mkdir_recurse(const char *dirname)

{

  size_t counter;

  char *nextdir;

  int ret;

  nextdir = strdup(dirname);

  /* skip preficing slashes */

  for(counter = 0; nextdir[counter] == '/'; counter ++)

    ;

  for(; nextdir[counter]; counter ++)

    {

      /** @TODO OS-portabalize the path-separators */

      if(nextdir[counter] == '/')

        {

          nextdir[counter] = '\0';

          ret = mkdir(nextdir, S_IRWXU | S_IRGRP | S_IROTH);

	  if(ret == -1

	     && errno != EEXIST)

	    {

	      perror("mkdir");

	      free(nextdir);

	      return 1;

	    }

          nextdir[counter] = '/';

        }

    }

  ret = mkdir(nextdir, S_IRWXU | S_IRGRP | S_IROTH);

  if(ret == -1

     && errno != EEXIST)

    {

      perror("mkdir");

      free(nextdir);

      return 1;

    }

  free(nextdir);

  return 0;

}

/**

 @TODO: Use for constructing path to job data locally and/or on data.distren.org

 */

int job_build_path(char *filename, unsigned int jobnum)

{

  return 0;

}

int downloadTar(char *url, char *destinationPath){

  // Prepare to download the job tar if it isn't  present

  struct stat buffer;

  int fstatus = stat(destinationPath, &buffer);

  if(fstatus == -1)

    {

      // Download the Tar

      int ret = curlget(url, destinationPath);

      if(ret == 0){

        fprintf(stderr, "Job data retrieved successfully\n");

        // free(url); @FIXME: causes doublefree! Curl must free the url?

        return 0;

      }

      else if(ret == 1){

        fprintf(stderr, "Downloading job data from %s failed. Check your network connection.\n",url);

        free(url);

        return 1; // Eventually make a retry loop

      }

    }

  fprintf(stderr, "Tar at %s already exists! Download cancelled.\n", destinationPath);

  return 2;

}

int uploadOutput(char *pathtoOutput, char *urltoOutput, int jobnum, int framenum, int slavekey){

  //fprintf(stderr,"Uploading output %s to url %s for job %d and frame %d for slavekey %d", pathtoOutput, urltoOutput, jobnum, framenum, slavekey);

  if( !curlpost(pathtoOutput, urltoOutput, jobnum, framenum, slavekey)) // Uploads output

    {

      if(DEBUG)

        fprintf(stderr,"Upload successful, removing old output...\n");

      else

        fprintf(stderr,"Upload completed\n");

      remove(pathtoOutput); // Delete the file after its uploaded

      return 0;

    }

  else

    {

      fprintf(stderr,"Upload failed. Check your network connection. Retrying upload...\n");

      int tries=1;

      while(tries<=10 && curlpost(pathtoOutput, urltoOutput, jobnum, framenum, slavekey))

        {

          fprintf(stderr, "Upload failed. Trying again in 10 seconds... (attempt %d of 10)\n", tries);

          tries++;

          sleep(10);

        }

      return 1; // Upload failed after multiple tries

      // @FUTURE: Keep track of files that we were unable to upload, and upload them later

    }

}

void prepareJobPaths(int jobnum, int framenum, char *outputExt, char *datadir, char **urltoTar,char **pathtoTar,char **pathtoTardir,char **pathtoJob, char **pathtoJobfile, char **urltoJobfile, char **urltoOutput,char **pathtoOutput, char **pathtoRenderOutput, char **pathtoOutdir)

{

  // Variable Preparation

  char *jobdatapath;

   _distren_asprintf(&jobdatapath, "job%d", jobnum);

   _distren_asprintf(urltoTar, "http://data.distren.org/job%d/job%d.tar.gz", jobnum, jobnum); // Prepares URL to download from

   _distren_asprintf(urltoJobfile, "http://data.distren.org/job%d/job.blend", jobnum); // Prepares URL to download from

   _distren_asprintf(pathtoTar, "%s/%s/job%d.tar.gz", datadir, jobdatapath, jobnum); // Prepares destination to save to

   _distren_asprintf(pathtoTardir, "%s/%s/", datadir, jobdatapath); // Prepares destination to save to

   _distren_asprintf(pathtoJob, "%s/%s/", datadir, jobdatapath);

   _distren_asprintf(pathtoJobfile, "%s/%s/job.blend", datadir, jobdatapath ); // Prepares the path to the jobfile

   _distren_asprintf(pathtoRenderOutput, "%s/%s/output/job%d-frame#.%s", datadir, jobdatapath, jobnum, outputExt ); // Note: the # is for blender to put in framenum in output file

   _distren_asprintf(pathtoOutput, "%s/%s/output/job%d-frame%d.%s", datadir, jobdatapath, jobnum, framenum, outputExt ); // Note: the # is for blender to put in framenum in output file

   _distren_asprintf(pathtoOutdir, "%s/%s/output", datadir, jobdatapath);

   free(jobdatapath);

}

int checkUsername(char *username)

{

  if(username == NULL || strcmp(username, "!username") == 0 )

    {

     return 1;

   }

   else

     if( username != NULL || strcmp(username, "!username") != 0 )

       {

         // Log in the user

         if(login_user(username) == 1){

           // Logged in OK

           return 0;

         }