Key:

*-An important Item that needs to be finished

B-Initial of person who needs to fix this (Binki)

M-LordofWar needs to fix this

E-normaldotcom needs to fix this

x-finished task

=-Long-term goal

~-Cancelled goal

a-Goal needing assessment

?-needs reconsideration/explanation/rewording

Notes:

  Initial release will no longer feature SSH auth, distrend will handle auth telnet-style. User registration will be done via PHP.

  Simpleslave uses a PHP API to communicate with the mysql-backed server, allowing debugging while server software is nonfunctional.

Build

General

  =-documenation: manpages

  =-runtime support: should we eventually just go with APR, glib, maybe something from suckless people? Is libdaemon (or whatever's out there) useful?

  =-XML (XML-RPC): useful as a communication protocol? as RPC protocol? see atheme; useful as job description format possibly.

  B-remoteio: figure out how to handle communications; async + callbacks seems the best method to me atm

Master

  B/E/?-Add calls to remotio -> make API for sending messages ~= a function for every type of message defined in protocol.h (are SendSignal() and SendExtSignal() good enough?)

  a-XML-RPC interface for obtaining info

  =-charsets: the program should operate in UTF-8. This is partially required by libxml2's (and XML's) usage of UTF-8. We want the program's internal charset to be UTF-8 and figure out if we need w_char/whatever to fulfill UTF-8. I don't believe we do... mabye we just need a declaration that al data stored must be in UTF-8 format. -> we need to consider how charsets work when and if issues come up

  *-figure out delete-during-traversal with liblist

  *-after job is complete and verified by the user, tar frames for easy download

  x-Complete testing with interactive test mode

  x-Complete DB testing with php server workaround

  x-resolve conflict between liblist and libmysqlclient

  x-update frame assigner to distribute some frames for lower-priority jobs when there's a higher priority job (Ask LordOfWar if this is implemented in sprocs already(it is))

Slave

  B-UnpackJob() and supporting functions need to work

  E-Restore tar functionality after unpackJob() is functional

  E-Add calls to remoteio, once it works or even before it works -> write message API functions and API

  =-Make code more flexible for different types of jobs and operating systems (such as paths, libs, etc)

  =-Add code to allow pausing/resuming of the slave (including a running blender process) -> signal handling (SIGTSTP vs. SIGSTOP), distrenslave_control cmdline interface

  B-Add tmpdir variable (just like the datadir variable) that is compile-time or whatever -> simple API to allow configuration-file override while taking into account defaults set by ./configure

  =-Upload finished frames' output files while rendering subsequent frames

  x=Add lockout of slaves below a certain version

  x-Add simpleslave with code to work around nonexisting server codebase

  x-Backport SimpleSlave functionality to slave code

Options

  B-Push patch for relative includes in confuse to the confuse developers -- windows/mingw32 testing required first

  ?-Expand execio() to support stderr

  =-Rewrite some stuff, try to make it simpler

  =-Review all confuse interfacing

  x-Move server and client confuse code into their individual files, rather than in the common file. -- note: there wasn't any client code in options.c. There was the ``server'' section. That section is passed multiple times and provides information to remoteio on how to connect to servers

WebInterface

  *-Tarball validation and other checks

  *-Scale images on web interface, imagemagick?

  *-Add "stop" button functionality and other missing button functionality

  E-Check queries and output for all edit scripts, e.g., edit profile userlevel dropdown doesn't show the user's current userlevel

  E-Simplify web interface, make it more lightweight

  E-Make some icons for deleting/editing/etc that aren't horribly ugly

  E-Get rid of "My Output" or move some functionality to this area

  =-Allow "archiving" of jobs?

  =-Begin working on MeshLib seamless integration with DistRen

  =-Create install.php to guide through database connection and schema deployment

  =-Add feature to "purchase" priority with Render_Points

Overall

  *-Output type is currently a placeholder option at this point, make it actually affect the output

  *-Change the way the database handles storage of paths to frame data (path + extension?)

  *-Output resolution is currently a placeholder option, make it actually affect the output

  struct options_common *commonopts;

  /**

     initializations

  */

  datadir = NULL;

  server = NULL;

  username = NULL;

  password = NULL;

  hostname = NULL;

  char curopt;

  int runBenchmark = 0;

  while(((char)-1) != (curopt = getopt(argc, argv, "u:rh")))

     {

       if(curopt == ':')

         {

           fprintf(stderr, "-%c: is missing an argument\n", optopt);

           return 1;

         }

       else if(curopt == '?')

         {

           fprintf(stderr, "-%c: invalid option specified\n", optopt);

           return 1;

         }

       else if(curopt == 'h')

         {

           fprintf(stderr, "Usage: distrensimpleslave [option] \nStarts a distren slave\n\t-u\tset username (run after fresh install)\n\t-r\tRecalculate render power (benchmark)\n\t-h\tshow this help\n");

           return 2;

         }

       else if(curopt == 'r')

         {

           runBenchmark = 1;

           break;

         }

       else if(curopt == 'u')

         username = strdup(optarg);

         if(DEBUG)

           fprintf(stderr, "Putting username \"%s\" in distrenslave.conf\n", username);

         conf_replace("distrenslave.conf", "!username", username);

         fprintf(stderr, "Please invoke distrensimpleslave with no arguments to run with the username you just set\n");

         return 0;

     }

  /* Get conf data */

  options_init(argc, argv, &my_cfg, myopts, "slave", &commonopts);

  if(!datadir)

    {

      fprintf(stderr, "datadir not set\n");

      return 1;

    }

  if(!server)

    {

      fprintf(stderr, "server not set\n");

      return 1;

    }

  if(!username)

    {

      fprintf(stderr, "username not set\n");

      return 1;

    }

  if(!password)

    {

      fprintf(stderr, "password not set\n");

      return 1;

    }

  if(!hostname)

    {

      fprintf(stderr, "hostname not set\n");

      return 1;

    }

  /* Notifies the user if there no username in .conf */

  if(checkUsername(username))

    return 1;

  if(!strncmp(password, "!password",10))

    {

      fprintf(stderr, "You haven't specified a password. Please edit distrenslave.conf!\n");

      return 1;

    }

  // Variables needed for main loop

  int jobnum = 0;

  int framenum = 0;

  int slavekey = atoi(username); // @TODO: Make this more friendly

  char *urltoTar;      /* Full URL to the server-side location of job#.tgz */

  char *pathtoTar;     /* Full path to the location of the job#.tgz */

  char *pathtoTardir;

  // Temporary for uncompressed testing

  char *urltoOutput;   /* Full URL where output is posted */

  char *pathtoOutput;  /* Full path to the output (rendered) file */

  char *pathtoOutdir;  /* Full path to output directory */

  char *pathtoRenderOutput; /* Contains blender framenum placeholder */

  char *pathtoJob; /* Path to job data folder */

  char *pathtoJobfile; /* Full path to the job's main file */

  char *outputExt = "jpg";     /* Output Extension (e.g., JPG) */

  int haveWork = 0;

  int quit = 0;

  fprintf(stderr,"\nDistRen SimpleSlave Pre-Alpha %s\n- Experimental build: Use at your own risk!\n\n", PACKAGE_VERSION);

  int benchmarkTime = 0;

  int renderPower = 0;

  // @TODO: create _web function to force recalc if !isset($render_power) on server

  if(runBenchmark)

    {

      if(slaveBenchmark(datadir, &benchmarkTime, &renderPower))

	{

	  fprintf(stderr,"Benchmark failed! Exiting.\n");

	  return 1;

	}

      else

	{

	  fprintf(stderr,"Benchmark successful, time taken was %d seconds, giving you a render power of %d.\n",

		  benchmarkTime, renderPower);

	  _web_setrenderpower(slavekey, password, renderPower);

	  return 0;

	}

    }

  if(!DEBUG)

    fprintf(stderr, "Running..");

  // Main loop

  while(!quit)

    {

    // request work

    if(DEBUG)

      fprintf(stderr,"Requesting work from %s...\n", hostname);

    haveWork = _web_getwork(slavekey, password, &jobnum, &framenum);

    /* If we got a frame */

    if(haveWork)

      {

        fprintf(stderr,"Got work from server...\n");

        if(DEBUG)

          fprintf(stderr, "Preparing to render frame %d in job %d\n", framenum, jobnum);

        prepareJobPaths(jobnum, framenum, outputExt, datadir, &urltoTar, &pathtoTar, &pathtoTardir, &pathtoJob, &pathtoJobfile, &urltoJobfile, &urltoOutput, &pathtoOutput, &pathtoRenderOutput, &pathtoOutdir);

        //free(outputExt);

        mkdir(pathtoTardir, 0700);

        if(dlret == 0)

          fprintf(stderr,"Data retrieved successfully!\n");

        else if(dlret == 3){

          _web_resetframe(slavekey, password, jobnum, framenum);  // Unassign the frame on the server so other slaves can render it

          return 0; // ouput dir doesn't exist

        }

        else

          if(DEBUG)

        struct stat jbuffer;

        int jstatus = stat(pathtoJobfile, &jbuffer);

        if(jstatus == -1){

          fprintf(stderr,"Main job file does not exist, extracting...\n");

          // If error unpacking tarball

          if(unpackJob(pathtoJob, pathtoTar)){

            _web_resetframe(slavekey, password, jobnum, framenum);  // Unassign the frame on the server so other slaves can render it

            return 1;

          }

        }

        /* ignore return because directory may exist already */

        if(DEBUG)

          fprintf(stderr,"Creating output directory %s\n", pathtoOutdir);

        mkdir(pathtoOutdir, 0700);

        if(DEBUG)

          fprintf(stderr,"Marking frame started on server... ");

        _web_startframe(slavekey, password, jobnum, framenum);

        /* Execute blender */

        if(DEBUG){

          fprintf(stderr,"Executing blender on file %s\n", pathtoJobfile);

          fprintf(stderr,"Directing output to file %s\n", pathtoOutput);

        }

        if(exec_blender(pathtoJobfile, pathtoRenderOutput, framenum))

          {

            fprintf(stderr,"Error running Blender. Check your installation and/or your PATH.\n");

            _web_resetframe(slavekey, password, jobnum, framenum);  // Unassign the frame on the server so other slaves can render it

            return 1;

          }

        free(pathtoJobfile);

        pathtoJobfile = NULL;

        struct stat buffer;

        int fstatus = stat(pathtoOutput, &buffer);

        if(fstatus == -1){

          fprintf(stderr,"*** %s doesn't exist! Scene may not have camera, or your blender installation is not working.\n", pathtoOutput);

          _web_resetframe(slavekey, password, jobnum, framenum);  // Unassign the frame on the server so other slaves can render it

          return 1;

        }

        else{

          /* Post-execution */

          if(DEBUG)

            fprintf(stderr, "Finished frame %d in job %d, uploading...\n", framenum, jobnum);

          else

            fprintf(stderr,"Finished.\n");

          uploadOutput(pathtoOutput, urltoOutput, jobnum, framenum, slavekey); // @TODO: Handle return value

          free(urltoOutput);

          free(pathtoOutput);

          urltoOutput = NULL;

          pathtoOutput = NULL;

          // Tell the server that rendering and upload are complete

          _web_finishframe(slavekey, password, jobnum, framenum);

        }

      }

    else{

      if(DEBUG)

        fprintf(stderr,"No work to do. Idling...\n");

      else

        fprintf(stderr,".");

      sleep(300); // Poll every 300 seconds @TODO: remove polling

    }

    // @TODO: If the server says that every frame for the last jobnum is finished, OR if the data is getting old

    if(1 == 0)

      {

        // Note: individual frames are already deleted after uploading,

        // except for ones that couldn't be uploaded

        delete_jobdata(jobnum, datadir);

      }

  }

  free(my_cfg);

  free(outputExt);

  free(datadir);

  free(urltoTar);

  free(pathtoTar);

  free(pathtoTardir);

  free(pathtoJob);

  free(pathtoJobfile);

  free(urltoJobfile);

  free(urltoOutput);

  free(pathtoRenderOutput);

  free(pathtoOutdir);

  fprintf(stderr,"Goodbye!\n");

  return 0;

}

/*

  Copyright 2010 Nathan Phillip Brink, Ethan Zonca, Matthew Orlando

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "distrenjob.h"

#include "slavefuncs.h"

#include "common/asprintf.h"

#include "common/execio.h"

#include "common/protocol.h"

#include "common/remoteio.h"

#include <curl/curl.h>

#include <curl/easy.h>

#include <curl/types.h>

#include <archive.h>

#include <archive_entry.h>

#include <stdio.h>

#include <string.h>

#include <unistd.h>

#include <stdlib.h>

#include <sys/stat.h>

#include <fcntl.h>

#include <time.h>

#define DEBUG 0

/**

   Grabs the xml DOM node reached by an XPath.

   @param path an XPath that lead to DOM node

   @return the first node associated with the path or NULL if there is no match

 */

xmlNodePtr xml_quickxpath(xmlXPathContextPtr xpathctxt, xmlChar *path)

{

  xmlNodePtr toreturn;

  xmlXPathObjectPtr xmlxpathobjptr;

  xmlxpathobjptr = xmlXPathEval(path, xpathctxt);

  if(!xmlxpathobjptr

     || !xmlxpathobjptr->nodesetval->nodeNr)

    {

      fprintf(stderr, "XPath resolution failed for ``%s'' in ``%s'' (``%s'')\n", path, xpathctxt->doc->name, xpathctxt->doc->URL);

      return (xmlNodePtr)NULL;

    }

  toreturn = *(xmlxpathobjptr->nodesetval->nodeTab);

  xmlXPathFreeObject(xmlxpathobjptr);

  return toreturn;

}

int delete_jobdata(int jobnum, char *datadir)

{

  char *jobpath;

  _distren_asprintf(&jobpath, "%s/%d", datadir, jobnum);

  return 0;

}

/** Function referenced by curlget() to write data to disk. */

size_t curl_writetodisk(void *ptr, size_t size, size_t nmemb, FILE *stream)

 {

    return fwrite(ptr, size, nmemb, stream);

  }

/** Helper function for cURL's progress display */

int curl_progress(char *Bar, double t, double d, double ultotal, double ulnow)

{

fprintf(stderr,"Downloading: %f%% complete\r",d/t*100);

return 0;

}

/** Retrieves a URL with cURL and saves it to disk */

CURLcode curlget(char *url, char *out){

  fprintf(stderr,"Preparing to download %s\n",url);

  double *Bar; // Stores cURL progress display info

  CURL *curl;

  CURLcode res;

  FILE *outfile;

  res = CURLE_FAILED_INIT;

  curl = curl_easy_init();

  if(curl)

    {

    outfile = fopen(out, "w"); // Open where we're writing to

    if(outfile == NULL)

      return 2; // Output dir doesn't exist

    curl_easy_setopt(curl, CURLOPT_URL, url);

    curl_easy_setopt(curl, CURLOPT_WRITEDATA, outfile);

    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curl_writetodisk); // this MUST be set for win32 compat.

    curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);

    curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, (curl_progress_callback)&curl_progress);

    curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, &Bar);

    res = curl_easy_perform(curl);

    curl_easy_cleanup(curl);

    fclose(outfile);

  }

  fprintf(stderr,"\n"); // Clears out he progressbar's carriage return

  return res; // 0 is OK, 1 is 404 or other error

}

/** Posts a file to a url with cUrl */

CURLcode curlpost(char *filename, char *url, int jobnum, int framenum, int slavekey)

{

  char *targetname = "uploadedfile"; // Name of the target in the php file on the server (Don't change me unless you have different PHP code)

  CURL *curl;

  CURLcode res;

  struct curl_httppost *formpost=NULL;

  struct curl_httppost *lastptr=NULL;

  struct curl_slist *headerlist=NULL;

  static const char buf[] = "Expect:";

  char *sjobnum;

  char *sframenum;

  char *sslavekey;

  if(DEBUG)

    fprintf(stderr,"Uploading to %s\n", url);

  _distren_asprintf(&sjobnum,"%d",jobnum);

  _distren_asprintf(&sframenum,"%d",framenum);

  _distren_asprintf(&sslavekey,"%d",slavekey);

  curl_global_init(CURL_GLOBAL_ALL);

  /* jobnum field... */

   curl_formadd(&formpost,

                &lastptr,

                CURLFORM_COPYNAME, "jobnum",

                CURLFORM_COPYCONTENTS, sjobnum, CURLFORM_END);

  /* framenum field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "framenum",

               CURLFORM_COPYCONTENTS, sframenum, CURLFORM_END);

  /* slavekey field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "slavekey",

               CURLFORM_COPYCONTENTS, sslavekey, CURLFORM_END);

  /* upload field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, targetname,

               CURLFORM_FILE, filename,

               CURLFORM_END);

  /* filename field... */

  curl_formadd(&formpost,

               &lastptr,

               CURLFORM_COPYNAME, "filename",

               CURLFORM_COPYCONTENTS, filename,

               CURLFORM_END);

  fprintf(stderr,"Downloading benchmark data...\n");

  curlget("http://data.distren.org/benchmark.blend", input); // Download to input file location

  char *command = "blender"; // @TODO: We currently expect this to be in PATH

  char *cmd[] = { command, "-b", input, "-o", output, "-f", frame_str, "-t", "0", (char *)NULL }; // arguments for blender

  // fprintf(stderr,"Preparing to execute command: %s -b %s -o %s -f %s\n", command, input, output, frame_str);

  fprintf(stderr,"Running benchmark...\n");

  long startTime;

  long endTime;

  time(&startTime);

  char buf[20];

  struct execio *testrem;

  size_t readlen;

  ret = execio_open(&testrem, command, cmd);

  if(ret)

    {

      fprintf(stderr, "Error executing blender\n");

      return 1;

    }

  buf[19] = '\0';

  while(!execio_read(testrem, buf, 19, &readlen))

    {

       buf[readlen] = '\0';

       fprintf(stderr, "read \"%s\"\n", buf);

    }

  execio_close(testrem);

  time(&endTime);

  struct stat buffer;

  int ostatus = stat(realOutput, &buffer);

  if(ostatus == -1){

    ret = 1; // Return error if output wasn't generated

  }

  else

    remove(output);

  *benchmarkTime = abs(difftime(startTime,endTime));

  float tmp = *benchmarkTime;

  tmp = (1/tmp) * 50000;

  *renderPower = (int)tmp;

free(realOutput);

  free(frame_str);

  free(input);

  free(output);

  return ret;

}

/** *********************************************************************************************************/

/** Why hello ohnobinki! Normaldotcom has graciously prepared this portion of code for you to work on! Yay! */

/** *********************************************************************************************************/

/**

   Extracts archive to the specified directory, creating this directory

   if necessary (but this directory will not be recursively created).

   @param outdir output directory

   @param pathtoTar filename of the archive to extract

   ohnobinki: please make me work :-\

 */

int unpackJob(char *outdir, char *pathtoTar)

{

  int ret;

  struct archive *a;

  struct archive_entry *ae;

  int astatus;

  /* ignore return because directory may exist already */

  mkdir(outdir, 0700);

  ret = chdir(outdir);

  if(ret == -1)

    {

      perror("chdir");

      return 1;

    }

  a = archive_read_new();

  ae = archive_entry_new();

  archive_read_support_compression_all(a);

  archive_read_support_format_raw(a);

  astatus = archive_read_open_filename(a, pathtoTar, 8192);

  if (astatus != ARCHIVE_OK)

    {

      return 1;

    }

  for(astatus = ARCHIVE_OK;

      astatus == ARCHIVE_OK

        || astatus == ARCHIVE_WARN;

      )

    {

      astatus = archive_read_next_header2(a, ae);

      if(astatus == ARCHIVE_WARN)

      if(astatus == ARCHIVE_OK

         || astatus == ARCHIVE_WARN)

        astatus = archive_read_extract(a, ae,

                                       ARCHIVE_EXTRACT_NO_OVERWRITE

                                       | ARCHIVE_EXTRACT_SECURE_SYMLINKS

                                       | ARCHIVE_EXTRACT_SECURE_NODOTDOT);

    }

  archive_entry_free(ae);

  archive_read_finish(a);

  if(astatus != ARCHIVE_EOF)

    {

      return 1;

    }

  return 0;

}

/** Logs the user into the server after ensuring that keys exist

    ohnobinki: I assume you could use this for remoteio, or just kill it

*/

int login_user(char *username)

{

  // @TODO: Put some telnet-style auth code here unless this is obselete

  return 1; // success

}

/**

   Sends the server a single request (see protocol.h)

   ohnobinki: This should hopefully work, maybe ;D

*/

int sendSignal(struct remoteio *rem, char signal)

{

  size_t written;

  size_t towrite;

  char *ssignal;

  _distren_asprintf(&ssignal, "%c", signal);

  towrite = strlen(ssignal);

  while( towrite

         && !remoteio_write(rem, ssignal, towrite, &written))

    {

      fprintf(stderr, "Sending request...\n");

      towrite -= written;

    }

  if(written)

    return 0;

  /**

     if remoteio_write returned 1, the connection

     is probably dead or there was a real error

   */

  return 1;

}

/**

   Sends the server an extended signal (request + data)

   ohnobinki: I have no clue how you really want to handle this. Please clarify/edit

*/

int sendExtSignal(struct remoteio *rem, char signal, char *data){

  size_t written;

  size_t towrite;

  char *ssignal;

  _distren_asprintf(&ssignal, "%c%s", signal, data); // Just append the data FIXME: We should do this differently

  towrite = strlen(ssignal);

  while( towrite

          && !remoteio_write(rem, ssignal, towrite, &written))

     {

       fprintf(stderr, "Sending request...\n");

       towrite -= written;

     }

   if(written)

     return 0;

   /**

      if remoteio_write returned 1, the connection

      is probably dead or there was a real error

    */

   return 1;

}

/* Port of web functions for standard code

   Currently, most functions are stubs due to lack

   of socket reading code

   ohnobinki: I can take care of a fair amount of this, but the remotio reading and writing is where you should really lay down some code.

*/

/** marks frame finished on server */

void finishframe(struct remoteio *rem, int jobnum, int framenum){

  char* data;

  _distren_asprintf(&data, "%d%d", jobnum, framenum);

  sendExtSignal(rem, DISTREN_REQUEST_DONEFRAME, data);

}

/** resets frame to unassigned on server */

void resetframe(struct remoteio *rem, int jobnum, int framenum){

  fprintf(stderr,"Resetting frame %d in job %d on server... ",framenum,jobnum);

  char* data;

  _distren_asprintf(&data, "%d%d", jobnum, framenum);

 sendExtSignal(rem, DISTREN_REQUEST_RESETFRAME, data);

}

/** marks frame assigned on server */