DistRen is a joint initiative of [protofusion] (http://protofusion.org/) and Oh! No! Publishing (http://ohnopublishing.net/)

The principal devs are:

Nathan Phillip Brink <ohnobinki@ohnopublishing.net>

2009-01-04  Nathan Phillip Brink  <ohnobinki@gmail.com>

	* Initial construction of an autoconfy filestructure.

Key:

*-An important Item that needs to be finished

B-Initial of person who needs to fix this (Binki)

M-LordofWar needs to fix this

E-normaldotcom needs to fix this

x-finished task

=-Long-term goal

?-needs reconsideration/explanation/rewording

Notes:

  Initial release will no longer feature SSH auth, distrend will handle auth telnet-style. User registration will be done via PHP.

Build

General

=-documenation: manpages

=-runtime support: should we eventually just go with APR, glib, maybe something from suckless people? Is libdaemon (or whatever's out there) useful?

=-XML (XML-RPC): useful as a communication protocol? as RPC protocol? see atheme; useful as job description format possibly.

*-remoteio: figure out how to handle communications; async + callbacks seems the best method to me atm

Master

*=Add lockout of slaves below a certain version

*-Complete testing with interactive test mode

 -update frame assigner to distribute some frames for lower-priority jobs when there's a higher priority job

B-Add calls to remotio -> make API for sending messages ~= a function for every type of message defined in protocol.h

 -Write a stub for getting info from the tarball/validifying the tarball. Read distren-job.xml, a file in the tarball, to find out 1. which rendering system to use (that system, e.g. blender/povray, can read more specifics, such as name of file to pass to blender and frames. Options common between different systems will be handled in common as best as possible)

 -Write a stub for publishing file and constructing job description so that the job can be shared

B-XML-RPC interface for obtaining info

=-charsets: the program should operate in UTF-8. This is partially required by libxml2's (and XML's) usage of UTF-8. We want the program's internal charset to be UTF-8 and figure out if we need w_char/whatever to fulfill UTF-8. I don't believe we do... mabye we just need a declaration that al data stored must be in UTF-8 format. -> we need to consider how charsets work when and if issues come up

M-explain members of struct general_info; e.g., what is timestamp used for, what's the point of the hibernate var, etc.

*-resolve conflict between liblist and libmysqlclient

*-figure out delete-during-traversal for liblist

Slave

 -Fix current bugs

*-Add calls to remoteio, once it works or even before it works -> write message API functions and API

=-Make code more flexible for different types of jobs and operating systems (such as paths, libs, etc)

=-Add code to allow pausing/resuming of the slave (including a running blender process) -> signal handling (SIGTSTP vs. SIGSTOP), distrenslave_control cmdline interface

B-Add tmpdir variable (just like the datadir variable) that is compile-time or whatever -> simple API to allow configuration-file override while taking into account defaults set by ./configure

*-Finish stub to delete old job data and output

=-Upload finished frames' output files while rendering subsequent frames

Options

=-Rewrite some stuff, try to make it simpler

x-Move server and client confuse code into their individual files, rather than in the common file. -- note: there wasn't any client code in options.c. There was the ``server'' section. That section is passed multiple times and provides information to remoteio on how to connect to servers

=-Review all confuse interfacing

B-Push patch for relative includes in confuse to the confuse developers -- windows/mingw32 testing required first

?-Expand execio() to support stderr

#

# This file is a part of DistRen.

#

# Distren is free software: you can redistribute it and/or modify

# it under the terms of the GNU Affero General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# DistRen is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU Affero General Public License for more details.

#

# You should have received a copy of the GNU Affero General Public License

# along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

AC_PREREQ(2.61)

AC_INIT([distren],[0.0],[http://bugs.ohnopub.net/])

AC_CONFIG_SRCDIR([src/server/distrend.c])

AC_CONFIG_MACRO_DIR([m4])

AC_PROG_CC

AC_PROG_LIBTOOL

#AC_PROG_RANLIB #don't add this even if autoscan says to, because AC_PROG_LIBTOOL is enough

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/* This code is meant to submit files to the distren server(s). Let's say server for now. And let's state file purpose in each file just so we don't get screwed up! */

/* Ideas for php-side stuff:

 *

 * PHP adds the users... has captcha and email confirmation wth a pseudo-random number that

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef DISTREN_H

#define DISTREN_H 1

/*

  Public API for DistRen

 */

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/*

  Implementation of distren_* functions from distren.h excluding distren_job_* functions.

 */

#include "libdistren.h"

#include <malloc.h>

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef LIBDISTREN_H_

#define LIBDISTREN_H_ 1

/*

  Private definitions for libdistren.

 */

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/*

  Functions and code for retrieving the client's configuration information.

 */

#include "client/libdistren.h"

#include "common/options.h"

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/*

  Implementation of distren_job_* functions from distren.h.

 */

#include "libdistren.h"

/**

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/*

  Implementation of libdistren functions that exist to prevent this library from being biased toward certain methods of reporting errors/debug info or free()ing and malloc()ing

 */

#include "libdistren.h"

void *_malloc(distren_t distren, size_t size)

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef _DISTREN_EXECIO_H

#define _DISTREN_EXECIO_H

/*

  This file tries to abstract away getting a socket/fd that talks to a spawned program

 */

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef _DISTREN_LIBREMOTEIO_H

#define _DISTREN_LIBREMOTEIO_H

#include "remoteio.h"

#include <confuse.h>

/**

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "options.h"

#include "common/asprintf.h"

#include "common/misc.h"

#include "common/libremoteio.h"

#include "common/protocol.h"

/*

  This file is a part of distren.

  Distren is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  Distren is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with distren.  If not, see <http://www.gnu.org/licenses/>.

*/

/*

  The point of this file is to handle options parsing that is common to both the server and client of distren. Options parsing that is specific to either should be managed by the server or the client. This means that the user of this interface will be given a handle to a part of a confuse config tree and have to eat that part of the tree to fulfill its own options.

  Unfortunately, this extra layer of abstraction shall possibly require a common_opts struct to be made that must be passed to all functions that use the common config. This would include any interface in /src/common/

 */

#ifndef _DISTREN_OPTIONS_H

#define _DISTREN_OPTIONS_H

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "protocol.h"

#include <malloc.h>

#define DISTREN_REQUEST_MAGIC (0x32423434)

int distren_request_new(struct distren_request **req, uint32_t len, enum distren_request_type type)

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef DISTREN_PROTOCOL_H

#define DISTREN_PROTOCOL_H

#include <stdint.h>

/**

   Server types:

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "libremoteio.h"

#include "execio.h"

#include "asprintf.h"

#include <errno.h>

#ifndef _WIN32

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef _DISTREN_REMOTEIO_H

#define _DISTREN_REMOTEIO_H

#include <stdlib.h>

/*

  RemoteIO provides an abstraction to the method of talking to a remote distrend. It is a layer on top of execio that should provide an equivalent interface.

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

/* This file contains the code which both processes (renders) jobs as a slave, and the code which distributes frames to slaves after receiving them from the client portion of the codebase. */

#include "distrenjob.h"

#include "listen.h"

#include "slavefuncs.h"

#include "mysql.h"

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

struct distrend_config;

#ifndef _DISTREN_DISTREN_H_

#define _DISTREN_DISTREN_H_

#include <confuse.h>

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "distrenjob.h"

#include "slavefuncs.h"

#include "common/asprintf.h"

#include <libxml/parser.h>

#include <libxml/tree.h>

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef _DISTREN_DISTRENJOB_H

#define _DISTREN_DISTRENJOB_H

/**

   This file stores the distrenjob and frameset structs and prototypes for some functions to manipulate/use these.

 */

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "listen.h"

#include <errno.h>

#include <fcntl.h>

#include <list.h>

#include <malloc.h>

#include <netinet/in.h>

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

struct distrend_clientset;

struct distrend_listen;

struct distrend_client;

#ifndef _DISTREN_LISTEN_H

#define _DISTREN_LISTEN_H

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "mysql.h"

#include <mysql/mysql.h>

#include "common/asprintf.h"

#include "common/protocol.h"

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#ifndef MYSQL_H_

#define MYSQL_H_

#include "distrenjob.h"

/*

  This file is a part of DistRen.

  DistRen is free software: you can redistribute it and/or modify

  it under the terms of the GNU Affero General Public License as published by

  the Free Software Foundation, either version 3 of the License, or

  (at your option) any later version.

  DistRen is distributed in the hope that it will be useful,

  but WITHOUT ANY WARRANTY; without even the implied warranty of

  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

  GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License

  along with DistRen.  If not, see <http://www.gnu.org/licenses/>.

*/

#include "slavefuncs.h"