It is very strongly recommended that your school.d entry provide a

  crawling interface. This is necessary for the course_id

  autocompletion functionality. It may be extended to be used for

  other purposes later too.

  When writing a crawler interface, it is essential that a

  publicly-accessible data resource be used. Anyone should be able to

  get a copy of slate_permutate and use it to generate autocompletion

  data for your school. You may not use resources or backends which

  are password-restricted since that gives you a monopoly on providing

  slate_permutate for your particular school. Of course, if you want

  to do such a thing you may, but you must publish the source-code

  used to do so (as this project is protected by the Affero GPL) (but

  that doesn't mean you need to publish your password). Just, such a

  school backend module will never be accepted into mainline

  slate_permutate since only those with the proper access privileges

  can maintain such a backend, etc.

  The current crawler API has two steps. First,

  <school_id>_crawl_semester_list() is called to retrieve a list of

  crawlable semesters for the school. Then

  <school_id>_crawl_semester_get() is called once for each semester.

  - $school: The school handle for <school_id>.

  Returns 0 on success and nonzero on failure.

  This function should scrape the school's website and build a list of

  scrapable semesters. For each semester, it should instantiate a

  Semester object with metadata about that semester and then return an

  array of these Semester objects.

  - $school: The school handle for <school_id>

  - $semester: The semester object to which courses are added.

  Returns 0 on success and otherwise on failure.

  This function is to add courses (or course_slots) to the passed

  Semester object so that every available course for the given

  semester is catalogued by the Semester object. Upon successful

  return of this function (indicated by returning 0), the caller will

  store the data to disk for use by the autocomplete engine.