SlatePermutate Changelog /inc/school.crawl.inc

Changelog - /inc/school.crawl.inc @ 1838a6f6fa20 - showing 23 out of 23 revisions

binki	1838a6f6fa20	13 years ago	Update Hope College crawler to be stream/chunk based, lowering its memory usage. default
binki	490910f07d79	13 years ago	Merge in partially-working half-semester support. default
binki	3487a5b3cbfd	13 years ago	Fix some crawler utility functions to handle relative URI resolving correctly and be more flexible when parsing days. default
binki	b5c4b3fbe060	13 years ago	Add support for Dordt College. default
binki	1cf1f641a5e5	14 years ago	Make school_crawl_url() actually function properly for relative URIs. default
binki	da3582f113b9	14 years ago	Add support for Sunday, fixing bug #98 . default
binki	c9c12e4e603a	14 years ago	Set the User-Agent header in our crawler. default
binki	556c9319aa65	14 years ago	Add a CSV parser which works with the sorts of CSV we need to work with more than PHP's str_getcsv() or fgetcsv() functions. default
binki	dcc324bd004e	14 years ago	Fix some form parsing functions in the crawler API. default
binki	3db76bd5a41c	14 years ago	Refactor crawling to write out data a semester at a time instead of for all semesters at once, enabling PHP to use a reasonable memory limit. Convert calvin, cedarville, and ccbcmd to the new crawling method. default
binki	43acd1a78fa7	14 years ago	Make PHP files more emacs-friendly. default
binki	775e75832d2e	14 years ago	Support multiple section meetings for school_id=calvin. I was warned about this by Tom Graham but I never listened ;-). Fixes bug 109. default
binki	6583b0529782	15 years ago	Fix the school_crawl_days_format() school crawler support function to log errors through school_crawl_logf() instead of error_log(). default
binki	f59baf5fa7dc	15 years ago	Support crawling a single semester of umich. Closes bug 64. default
binki	977f2f6ae844	15 years ago	Fix some incorrect calls to stricmp() which should instead call strcasecmp(). default
binki	fcf10ab116db	15 years ago	New functions which may be useful for crawlers. default
binki	b5e0a0fe027d	15 years ago	Use an internal interface for logging information when crawling a school. default
binki	3b78fdf04ce4	15 years ago	Support multiple semesters. Changed crawler API to accept an array to which semesters are appended instead of passing the crawlers an already-created semester object to fill. default
binki	f19ed2cbeb20	15 years ago	Add school ccbcmd (The Community College of Bultimore County) with crawler. Also, add additional crawler utility functions and add Saturday support to the crawling utility functions. default
binki	9fdbdf39b43b	15 years ago	Move geturi() from calvin's crawler so that all crawlers can use it, renamed to school_crawl_geturi(). default
binki	12553a2740cb	15 years ago	Split out the Section class into Section and SectionMeeting. This allows the flexibility of Cedarville's system where a single section may have multiple meeting times -- most commonly for labs, but sometimes classes have multiple lectures. default
binki	1d417d9e6bb3	15 years ago	Cedarville crawler converted to use the generalized crawling interface; it now populates the Semester object with actual section data. Cedarville has some strange course constrainst, however, which would require changes to the core algorithm. default
binki	6cb196f112d9	15 years ago	A school website-crawling infrastructure. Supports crawling Calvin's website and producing JSON for jqueryui's autocomplete functionality. Also creates a JSON description of the list sections for each course, awaiting JS-support for AJAX section autocreation. default