Installation Instructions

*************************

Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,

   This file is free documentation; the Free Software Foundation gives

unlimited permission to copy, distribute and modify it.

Basic Installation

==================

   Briefly, the shell commands `./configure; make; make install' should

configure, build, and install this package.  The following

more-detailed instructions are generic; see the `README' file for

instructions specific to this package.

   The `configure' shell script attempts to guess correct values for

various system-dependent variables used during compilation.  It uses

those values to create a `Makefile' in each directory of the package.

It may also create one or more `.h' files containing system-dependent

definitions.  Finally, it creates a shell script `config.status' that

you can run in the future to recreate the current configuration, and a

file `config.log' containing compiler output (useful mainly for

debugging `configure').

   It can also use an optional file (typically called `config.cache'

and enabled with `--cache-file=config.cache' or simply `-C') that saves

the results of its tests to speed up reconfiguring.  Caching is

disabled by default to prevent problems with accidental use of stale

cache files.

   If you need to do unusual things to compile the package, please try

to figure out how `configure' could check whether to do them, and mail

diffs or instructions to the address given in the `README' so they can

be considered for the next release.  If you are using the cache, and at

some point `config.cache' contains results you don't want to keep, you

may remove or edit it.

   The file `configure.ac' (or `configure.in') is used to create

`configure' by a program called `autoconf'.  You need `configure.ac' if

you want to change it or regenerate `configure' using a newer version

of `autoconf'.

The simplest way to compile this package is:

  1. `cd' to the directory containing the package's source code and type

     `./configure' to configure the package for your system.

     Running `configure' might take a while.  While running, it prints

     some messages telling which features it is checking for.

  2. Type `make' to compile the package.

  3. Optionally, type `make check' to run any self-tests that come with

     the package.

  4. Type `make install' to install the programs and any data files and

     documentation.

  5. You can remove the program binaries and object files from the

     source code directory by typing `make clean'.  To also remove the

     files that `configure' created (so you can compile the package for

     a different kind of computer), type `make distclean'.  There is

     also a `make maintainer-clean' target, but that is intended mainly

     for the package's developers.  If you use it, you may have to get

     all sorts of other programs in order to regenerate files that came

     with the distribution.

  6. Often, you can also type `make uninstall' to remove the installed

     files again.

Compilers and Options

=====================

   Some systems require unusual options for compilation or linking that

the `configure' script does not know about.  Run `./configure --help'

for details on some of the pertinent environment variables.

   You can give `configure' initial values for configuration parameters

by setting variables in the command line or in the environment.  Here

is an example:

     ./configure CC=c99 CFLAGS=-g LIBS=-lposix

   *Note Defining Variables::, for more details.

Compiling For Multiple Architectures

====================================

   You can compile the package for more than one kind of computer at the

same time, by placing the object files for each architecture in their

own directory.  To do this, you can use GNU `make'.  `cd' to the

directory where you want the object files and executables to go and run

the `configure' script.  `configure' automatically checks for the

source code in the directory that `configure' is in and in `..'.

   With a non-GNU `make', it is safer to compile the package for one

architecture at a time in the source code directory.  After you have

installed the package for one architecture, use `make distclean' before

reconfiguring for another architecture.

   On MacOS X 10.5 and later systems, you can create libraries and

executables that work on multiple system types--known as "fat" or

"universal" binaries--by specifying multiple `-arch' options to the

compiler but only a single `-arch' option to the preprocessor.  Like

this:

     ./configure CC="gcc -arch i386 -arch x86_64 -arch ppc -arch ppc64" \

                 CXX="g++ -arch i386 -arch x86_64 -arch ppc -arch ppc64" \

                 CPP="gcc -E" CXXCPP="g++ -E"

   This is not guaranteed to produce working output in all cases, you

may have to build one architecture at a time and combine the results

using the `lipo' tool if you have problems.

Installation Names

==================

   By default, `make install' installs the package's commands under

`/usr/local/bin', include files under `/usr/local/include', etc.  You

can specify an installation prefix other than `/usr/local' by giving

`configure' the option `--prefix=PREFIX'.

   You can specify separate installation prefixes for

architecture-specific files and architecture-independent files.  If you

pass the option `--exec-prefix=PREFIX' to `configure', the package uses

PREFIX as the prefix for installing programs and libraries.

Documentation and other data files still use the regular prefix.

   In addition, if you use an unusual directory layout you can give

options like `--bindir=DIR' to specify different values for particular

kinds of files.  Run `configure --help' for a list of the directories

you can set and what kinds of files go in them.

   If the package supports it, you can cause programs to be installed

with an extra prefix or suffix on their names by giving `configure' the

option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.

Optional Features

=================

   Some packages pay attention to `--enable-FEATURE' options to

`configure', where FEATURE indicates an optional part of the package.

They may also pay attention to `--with-PACKAGE' options, where PACKAGE

is something like `gnu-as' or `x' (for the X Window System).  The

`README' should mention any `--enable-' and `--with-' options that the

package recognizes.

   For packages that use the X Window System, `configure' can usually

find the X include and library files automatically, but if it doesn't,

you can use the `configure' options `--x-includes=DIR' and

`--x-libraries=DIR' to specify their locations.

Particular systems

==================

   On HP-UX, the default C compiler is not ANSI C compatible.  If GNU

CC is not installed, it is recommended to use the following options in

order to use an ANSI C compiler:

and if that doesn't work, install pre-built binaries of GCC for HP-UX.

   On OSF/1 a.k.a. Tru64, some versions of the default C compiler cannot

parse its `<wchar.h>' header file.  The option `-nodtk' can be used as

a workaround.  If GNU CC is not installed, it is therefore recommended

to try

     ./configure CC="cc"

and if that doesn't work, try

     ./configure CC="cc -nodtk"

Specifying the System Type

==========================

   There may be some features `configure' cannot figure out

automatically, but needs to determine by the type of machine the package

will run on.  Usually, assuming the package is built to be run on the

_same_ architectures, `configure' can figure that out, but if it prints

a message saying it cannot guess the machine type, give it the

`--build=TYPE' option.  TYPE can either be a short name for the system

type, such as `sun4', or a canonical name which has the form:

     CPU-COMPANY-SYSTEM

where SYSTEM can have one of these forms:

   See the file `config.sub' for the possible values of each field.  If

`config.sub' isn't included in this package, then this package doesn't

need to know the machine type.

   If you are _building_ compiler tools for cross-compiling, you should

use the option `--target=TYPE' to select the type of system they will

produce code for.

   If you want to _use_ a cross compiler, that generates code for a

platform different from the build platform, you should specify the

"host" platform (i.e., that on which the generated programs will

eventually be run) with `--host=TYPE'.

Sharing Defaults

================

   If you want to set default values for `configure' scripts to share,

you can create a site shell script called `config.site' that gives

default values for variables like `CC', `cache_file', and `prefix'.

`configure' looks for `PREFIX/share/config.site' if it exists, then

`PREFIX/etc/config.site' if it exists.  Or, you can set the

`CONFIG_SITE' environment variable to the location of the site script.

A warning: not all `configure' scripts look for a site script.

Defining Variables

==================

   Variables not defined in a site shell script can be set in the

environment passed to `configure'.  However, some packages may run

configure again during the build, and the customized values of these

variables may be lost.  In order to avoid this problem, you should set

them in the `configure' command line, using `VAR=value'.  For example:

     ./configure CC=/usr/local2/bin/gcc

causes the specified `gcc' to be used as the C compiler (unless it is

overridden in the site shell script).

Unfortunately, this technique does not work for `CONFIG_SHELL' due to

an Autoconf bug.  Until the bug is fixed you can use this workaround:

     CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash

`configure' Invocation

======================

   `configure' recognizes the following options to control how it

operates.

`--help'

`-h'

     Print a summary of all of the options to `configure', and exit.

`--help=short'

`--help=recursive'

     Print a summary of the options unique to this package's

     `configure', and exit.  The `short' variant lists options used

     only in the top level, while the `recursive' variant lists options

     also present in any nested packages.

`--version'

`-V'

     Print the version of Autoconf used to generate the `configure'

     script, and exit.

`--cache-file=FILE'

     Enable the cache: use and save the results of the tests in FILE,

     traditionally `config.cache'.  FILE defaults to `/dev/null' to

     disable caching.

`--config-cache'

`-C'

     Alias for `--cache-file=config.cache'.

`--quiet'

`--silent'

`-q'

     Do not print messages saying which checks are being made.  To

     suppress all normal output, redirect it to `/dev/null' (any error

     messages will still be shown).

`--srcdir=DIR'

     Look for the package's source code in directory DIR.  Usually

     `configure' can determine that directory automatically.

`--prefix=DIR'

     Use DIR as the installation prefix.  *Note Installation Names::

     for more details, including other options available for fine-tuning

     the installation locations.

`--no-create'

`-n'

     Run the configure checks, but stop before creating any output

     files.

`configure' also accepts some other, not widely useful, options.  Run

`configure --help' for more details.

1. Create a CLI interface for distren

	X command: distren

	X args: -i infile.tar.bz2 -o outfile.tar.bz2 

	action: submits and blocks and retreives result of rendering the contents of infile.tar.bz2. Should currently just call stub functions.

2. Design of stubs in distren library:

   	X Stub for submitting file to server

	WIP Write configuration loader

	Stub for submitting file to a server (using a stream)

	Stub for waiting for server -> Maybe the client can just poll? I've got to decide this.

	Stub for retrieving result (using another stream)

3. Design of stubs in distrend:

	Stub for getting info from the tarball/validifying the tarball. Read distren-job.xml, a file in the tarball, to find out 1. which rendering system to use (that system, e.g. blender/povray, can read more specifics, such as name of file to pass to blender and frames. Options common between different systems will be handled in common as best as possible)

	Storage of data + metadata: Stubs for unpacking tarball, assigning the submission an ID, and retrieving the file by the submission ID.

	Actual rendering: Stub for calling subsystem's action that does the rendering. The subsystem should accept an XML fragment (a libxml subtree) that is specific to the subsystem. Ideally, subsystems will be both pluggable at both the dlopen() and link-time levels. dlopen may actually be easier than link-time pluggables...

	Stub for submission of new jobs to be distributed (a subsystem rendering a file with multiples frames should generate a subjob for each frame)

	Stub for publishing file and constructing job description so that the job can be shared

/*

  Copyright 2008 Nathan Phillip Brink, Ethan Zonca

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/* This file contains the code which both processes (renders) jobs as a slave, and the code which distributes frames to slaves after receiving them from the client portion of the codebase. */

 /* Just some notes -- Ethan Zonca

 * ++ Make data availible for other apps to parse

 * Server<==>Client Communication

 * Upload while rendering

 */

#include <stdio.h>

#include "execio.h"

// needs to be implemented

#include "options.h"

#include <confuse.h>

#define max 100

// maximum number of stored jobs in memory, per job type (lux/blend). Eventually we can dump this data to disk, or atleast the remainder not in memory...

int jobnum = 0;

int hcfjob; //highest consecutively finished job

int highest_jobnum; // The job number of the most recently created job, this is used when creating new jobs

// Structures for storing job information

// OOOOOOkay so we really need a struct for frameset rather than just an array, because we should track who each frame was sent out to. Please do this, someone! :D

struct {

  char *name;

  char *submitter;

  char *email;

  int priority;  // 1 is lowest, 10 is highest, 0 is done

  int percent_done;

  int completed_frames; // number of completed frames

  int total_frames; // how many frames are in the animation

  struct frameset **frameset; // What exactly is this now? hehe

  // For secondary upload location...

  char url;

  char login_name;

  char login_pass;

} blendjob[max];

// -- Matthew's handywork --

// struct for storing informaiton on each frame for a particular blender job

struct {

	int frame_num; // frame number to render

	char slave_name; // user that frame is assigned to

	int frame_status; // status of frame, 0= unassigned, 1= taken, 2= done

} frameset[frame_count]; // Frame count is the frame number if the animation starts on zero... but animations start at frame 1

                         // Using this method to save memory, because if animation starts on a high frame number, it would waste a lot of RAM on empty structures

// -- end of Matthew's handywork --

struct {

  char *name;

  char *submitter;

  char *email;

  int priority;

  int mode; // 0 = Static Render, stop at Spp, or infinity if spp is null. Framerange is ignored || 1 = animation, stop at Spp and stay in framerange.

  int spp;

  struct frameset ** frameset;

} luxjob[max];

int main(int argc, char *argv[])

{

/* !!!!!!! Important notes !!!!!!!!!!!!

So, we need a common key. Maybe. Or we need to generate a key in registeruser() and get it to the client somehow.

If we use a common key, then we'll need different passphrases for each user, which would be kinda crazy. How can

we get a key generated on the server, and transferred to the client's distrend?

*/

  int cont = 1;

  struct distrend_listenset *listenset;

  struct distrend_config *config;

  distrend_do_config(argc, argv, &config);

  distrend_listen(&listenset, config);

  /* This is called the ``main loop'' */

  while(cont)

    {

      struct distren_action *action;

      distrend_accept(&action);

      cont = distrend_do(action);

      distrend_action_free(action);

    }

  distrend_unlisten(listenset);

  distrend_config_free(config);

  return 0;

}

struct distrend_config

{

  cfg_t *mycfg;

  struct options_common *options;

  struct distrend_listen **listens; /*< Null terminated array of structs */

}

// Registration on server. Needs attention. (e.g., people could make tons of accounts with the key we give them...

// Set up distrend -c username email@example.com

// We must prevent multi-user-creation abuse, too.

void registeruser(char *username, char *email){

  // Logs into sandboxed user on zserver2 and registers a user. Should eventually generate a key on the server and return it to the user.

  // All created user accounts should be sandboxed accordingly, requiring a different skel, and the default shell to be rbash. They also

  // need a custom path, which can be defined in the .bash_profile of the skel. All activity of server-side configuration should be documented

  // to ease the creation of new distren servers, or it should be automated and made much more all-inclusive, not requiring much config.

  char buf[10];

  struct execio *testrem;

  char *execargv[] =

    {

  "ssh",

  "distren_setup@protofusion.org",

  "-i",

  "setup.rsa", // We gotta figure out how we're going to do the whole keys thing... maybe grab this key via http? But have a secret password compiled in distren to prevent fraud?

  "-p",

  "23",

  "sudo /usr/sbin/useradd",

  "-M",

  "-c",

  email,

  "-d",

  "/home/distren",

  "--gid",

  "541",

  username,

  (char *)NULL

  };

  size_t readlen;

  fprintf(stderr, "Opening stream:\n", execio_open(&testrem, "ssh", execargv));

  buf[9] = '\0'; // null-terminating the array...

  while(!execio_read(testrem, buf, 9, &readlen)) // What's with the readlen stuff?

    {

      if(readlen > 9) {

	fprintf(stderr, "!!!! Something is terribly wrong!\n");

      }

      if(buf == 0) {

	fprintf(stderr, "**** Operation successful, or so we hope. We got no output. Just kidding. You will never see this. Something else should catch this though! And registration should really have some sort of password.");

      }

    buf[readlen] = '\0'; // Null-terminating the end of it again based on how large the data is?

    fprintf(stderr, "read \"%s\"\n", buf);

    }

  execio_close(testrem);

}

void loginuser(char *username, int secret){

  char buf[10];

  struct execio *testrem;

  char *execargv[] =

    {

  "ssh",

  "distren_setup@protofusion.org",

  "-i",

  "setup.rsa", // We gotta figure out how we're going to do the whole keys thing... maybe grab this key via http? But have a secret password compiled in distren to prevent fraud?

  "-p",

  "23",

  (char *)NULL

  };

  size_t readlen;

  fprintf(stderr, "Opening stream:\n", execio_open(&testrem, "ssh", execargv));

  buf[9] = '\0'; // null-terminating the array...

  while(!execio_read(testrem, buf, 9, &readlen)) // What's with the readlen stuff?

    {

      if(readlen > 9) {

	fprintf(stderr, "!!!! Something is terribly wrong!\n");

      }

      if(buf == 0) {

	fprintf(stderr, "**** Operation successful, or so we hope. We got no output. Just kidding. You will never see this. Something else should catch this though!");

      }

    buf[readlen] = '\0'; // Null-terminating the end of it again based on how large the data is?

    fprintf(stderr, "read \"%s\"\n", buf);

    }

  execio_close(testrem);

}

/*

 frame[frame] Assignments:

  "0" - cancelled

  "1" - unassigned

  "2" - assigned to slave

  "3" - completed by slave and uploaded

 Have a script crawl through each job in the arrays, ordered by priority?, and assign a certain number of frames to each slave.

 Then we will need some sort of watchdog to monitor slaves on the main server to check for stales.

*/

/* This function creates frame array based on the total number of frames to be rendered, which will then be parsed by function frame_farmer. */

void frame_num_struct_builder(int sframe, int eframe) {

	int jobnum_new = highest_jobnum + 1;

	int total = sframe - eframe;

	int fcount = sframe; // Used to create all the frames in the structure from sframe to eframe

	int x = 0;

	blendjob[jobnum_new].total_frames = total; // sets the total number of frames in animation for status purposes

	while(x < total){

		blendjob[jobnum_new].frameset[x].frame_num = fcount;

		x++;

		fcount++;

	}

	highest_jobnum++;

}

// matches your computer up with a lovely frame to render

int frame_finder(){

	int finder_jobnum = 0;

	int frameset_count = 0; // the frameset number, because frames in an animation don't start at zero

	short int done = 0;

	for(int priority = 10; priority >= 1; priority--){ // start the scan for the next job with the highest priority

		finder_jobnum = hcfjob + 1; // reset it to start scanning at first uncompleted job for the pass at each priority level

		while(finder_jobnum <= highest_jobnum){

			if(blendjob[finder_jobnum].priority = priority){  // looks for a job with the current priority value

				done = 1;									  // notice it starts by looking at the oldest job first

				break;

			}

			if(done = 1)  // If it has found a job with the current priority value, it will break out of the loop

				break;    // If none is found it goes to the next job to see if it is of the current priority value

			else

				finder_jobnum++;

		}

		if(done = 1) // if job has been found, it lets it out of the priority changer loop

			break;

	}

	while(your_frame <= (sframe - eframe)){ // Finds the frameset number with a frame that needs to be rendered

		if (blendjob[finder_jobnum].frameset[frameset_count].frame_status = 0)  // If frame that is not assigned has been found, frameset_count is not changed

			break;																// and frameset_count is used to give the frame number later in this funciton

		x++;  // If frame is assigned or done, it goes to next frame

	}

blendjob[jobnum].frameset[x].frame_status++; // sets the value of the frame to 2, which means its taken

your_frame = blendjob[jobnum].frameset[x].frame_num; //  Takes the frameset found in the while statement above, and extracts the frame number from it and assigns it to the int your_frame

if(your_frame = null)  // If that job had no open frames for some reason, run the status report generator so that

	status_report_generator();  //the job priority can be changed to 0

return your_frame; // your_frame is returned as the frame to be rendered

}

// This figures out how much of the job is done, where jobnum corresponds to the job number

// This uses pointers, so when it is run it updates certain values in memory

void status_report_generator(){

	int num1 = hcfjob+1; // to scan through jobs

	int num2 = 0;		 // to scan through frames

	if(blendjob[num1].priority = 0)  //If the job after the highest consecutively finished job is finished

		hcfjob+1;  // adds 1 to the highest consecutively finished job

	while(num1 <= highest_jobnum){

		if(blendjob[num1].frameset[num2].priority != 0){ // If the job is not done, scan it

			int finished_frames = 0; // variable that counts the completed frames

			while(num2 <= blendjob[jobnum].total_frames){

				if(blendjob[jobnum].frameset[num2])

					finished_frames++;

				num2++;

			}

		}

		blendjob[num1].completed_frames = finished_frames;

		num1++;

	}

}

// This function makes the value of the frame 2, which means its completed.

void the_finisher(int frame){

	blendjob[jobnum].frameset[frame]++;

}

// Queuer: this function should add files to the queue

// Type: 0 = blender, 1 = luxrender

// jobnum is the current job number

void queue(int type, char *name, char *submitter, char *email, int priority, int mode, int spp, *frameset) {

  if(type == 1){

    blendjob[jobnum].name = name;

    blendjob[jobnum].submitter = submitter;

    blendjob[jobnum].email = email;

    blendjob[jobnum].priority = priority;

    blendjob[jobnum].frameset =  frameset;

  }

  if(type == 2){

    luxjob[jobnum].name = name;

    luxjob[jobnum].submitter = submitter;

    luxjob[jobnum].email = email;

    luxjob[jobnum].priority = priority;

    luxjob[jobnum].mode = mode;

    luxjob[jobnum].spp = spp;

    luxjob[jobnum].frameset = frameset;

    // handle lux modes somehow.

}

  else{

    // Throw error.

  }

jobnum++;

}

/*

Slave execution code lies below

Slave listens on server for a command in the format of each function...

We need if's for returns... ==> watchdog

*/

// Executors

void exec_blender(char *input, char *output, int frame) {

  int ret;

  /* SEGFAULTAGE :-D */

  char *cmd[] = { "blender", "-b", "-o", output, input, "-f", frame, (char *)0 };

  ret = execv("/usr/bin/blender", cmd);

  // OHNOBINKI! ... check this... Its supposed to send a command back to the server and run the_finisher(); function which sets the frame status to complete.

  fprintf(stdin, the_finisher(frame));

void exec_luxrender_single(char *input, char *output) {

  int ret;

  char *cmd[] = { "luxrender", "-something", "something", "something", (char *)0 };

  ret = execv("/usr/bin/luxrender", cmd);

}

void exec_luxrender_anim(char *input, char *output, int sframe, int eframe, int spp) {

// spp is samples per pixel limitation on each frame

  int ret;

  char *cmd[] = { "luxrender", "-something", "something", "something", (char *)0 };

  ret = execv ("/usr/bin/luxrender", cmd);

}

  return 0;

}