[Initial commit (mikeb@squaremobius.net--2004/fuschia--rdf--1) Michael Bridgen **20050128152426] [Tooling and NOTES towards using RDF (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-1) Michael Bridgen **20050109132249 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-1 Keywords: - Updated the implementation NOTES with some ideas on using RDF. - Separated out some bits of implementation and started on using RDF (broken). ] [Notes on RDF; start of implementation (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-2) Michael Bridgen **20050110181302 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-2 Keywords: Started on an RDF store that uses filesystem information. Also wrote some scattered notes on using RDF and some bits and pieces. ] [Calculated properties; Graphpath adapter; Notes (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-3) Michael Bridgen **20050112143810 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-3 Keywords: - Made the Graphpath adapter work sort of OK - Added a decorator storage that can calculate properties - Finished _loadObject for FilesystemStore - Notes on models and views; this is looking more zowie-ish - Semi-working overhaul of templates, much more to be done ] [Store and Graphpath adapter fixes for proper querying; templates (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-4) Michael Bridgen **20050113182832 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-4 Keywords: *Graphpath Adapter - RDF type gets initialiased per population, as per the graphpath examples *Store - Switched to using CONTAINS as the only semantic relation, rather than PARENT and the rest. It's simpler and makes more sense. - Wrap URIs in a Resource class, so we can tell what belongs as subjects and predicates and what doesn't - Add mtime to properties explicitly recorded - Fixed class (rdf:type) recording - Made the store properly lazy. It was failing reverse queries because it wasn't recording (say) reverse paths; sometimes this was because it saw the object there already from a previous subject query, and sometimes it was because the store hadn't been maintained properly. Now I distinguish between things put in the store so I can refer to them, and things explicitly loaded. It seems to work OK, but will have problems for reverse queries that the particular store doesn't know about. - Fixed some of the URL<->Path translation -- remove/append file extensions, check what things are before translating. *Templates - Added a very sketchy start to templates. Yuck. ] [Spit and polish to store and adapter (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-5) Michael Bridgen **20050118223200 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-5 Keywords: - Corrected relationship from .../parent to .../contains - Some additional checks for the type of resources in the store - Subject checks factored out (but some stragglers) - Consistent treatment of slashes in _toURL and _toPath; this was causing some problems with graphpath querying ] [Some refactoring and improvements to the store (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-6) Michael Bridgen **20050122170736 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-6 Keywords: - [code AbstractStore] -> [code BaseStore], which is more accurate - Tidied up and corrected some comments - Cache the file extension starting with a dot -- saves adding it later - Don't access a [code Resource]s uri directly outside that class - Improvements to [code _toPath] and [code _toURL] - Removed old CalculatedBlahBlah class -- I don't think it works anyway. - Enforce the URL format -- if a trailing /, treat it as a category In general: - More use of [code _isStory] and [code _isCategory], even when they duplicate effort, for the benefit of encapsulating them. ] [Added 'name' property for stories (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-7) Michael Bridgen **20050122174243 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-7 Keywords: - Normalise paths so that basename always gives the right answer - Put a 'name' property in for stories ] [A few notes and tidying up (mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-8) Michael Bridgen **20050128151419 mikeb@squaremobius.net--2004/fuschia--rdf--1--patch-8 Keywords: - Move resources into their own file -- perhaps can have a separate file for property definitions - Minor changes to notes and comments ] [Movement towards MVC -- stubs and notes Michael Bridgen **20050128163931] [Stub for the model Michael Bridgen **20050128174649] [Notes on version controlled and flavoured stores Michael Bridgen **20050209105433] [Add a federated store Michael Bridgen **20050215073146] [Notes from looking at Haystack Michael Bridgen **20050215081612] [Parameterise FilesystemStore to some extent Michael Bridgen **20050215130810 - Renamed FederatedStore -> AggreagatedStore and moved up in file - Made a story factory class, to parameterise the filesystem store with - Rationalised _is* predicates in filesystem store to be usable everywhere ] [Rename templates, remove stylesheet (it's in view) Michael Bridgen **20050222135911] [Blether in NOTES and TODO Michael Bridgen **20050228114328] [Easier syntax for resources Michael Bridgen **20050228114835] [Stylesheets with selectors for the view Michael Bridgen **20050228115145 - Implemented a stylesheet class which stores rules against selectors - A procedure for choosing the 'best' rule for a resource - Put a basic procedure for using the stylesheet in the view ] [Revised model; added shortcutting Union Michael Bridgen **20050303215343] [More notes and corrections Michael Bridgen **20050303215617] [Simplify view again -- don't need too much customising Michael Bridgen **20050303215739] [Make resource prefixes recursive so you can have, for example, fuschia.type.story Michael Bridgen **20050303222146] [Cleaned up, stubbed out View Michael Bridgen **20050303224350] [Move blank nodes out to resources, add function to find rdf:type Michael Bridgen **20050303224807] [Basic, basic controller Michael Bridgen **20050303225159] [Update TODO with immediate tasks Michael Bridgen **20050303230047] [Speculative code for index.cgi Michael Bridgen **20050306123248] [More notes, mostly refinement Michael Bridgen **20050601162355] [Beginning of r2, a fresh start Michael Bridgen **20051229112037 A start with some new abstractions. The focus will be on having multiple feeds, and a flexible template language (or languages), *and* doing a reasonable ting by default. I'm not so concerned with uniformity or reflection this time, but I think graphpath will be handy for templates. To this end, I've added a couple of Steps for sorting and restricting graphpath results by position. There's also some mixins for generic templates, which I'll make into usable classes once it's clear how the configuration works. ] [Move previous experiment to a subdirectory Michael Bridgen **20051212153616] [Add example feeds and some notes on the formats Michael Bridgen **20051230122112] [Filled out notes on the example feeds Michael Bridgen **20051230161235] [Notes on templates Michael Bridgen **20060103184612] [More notes on templates Michael Bridgen **20060103185644] [Elaborated on the syntax for CSS-like templates; notes on interpreting tempaltes Michael Bridgen **20060104160355] [A source that loads from the filesystem and a vocabulary Michael Bridgen **20051230021753 I've started a vocabulary, based on Atom with some schema information. It feels llike the schema information is going to help with rendering; for instance, the types of certain fields. The Atom schema also allows the content to be tagged. There's also a fairly specialised function for loading items from a filesystem, treating files as RFC822 messages and processing headers as item properties. ] [Example feed from suprglu.com Michael Bridgen **20051230203747] [Store feed information when loading from filesystem Michael Bridgen **20060102001959] [Current() node support Michael Bridgen **20060103232552 Operations in stylesheets typically refer to a context or current node; for supporting those, I've added a placeholder Graphpath step called Current(), which can be replaced in an expression with a particular node before evaluation. ] [Support for creating a store in the store abstraction Michael Bridgen **20060103233032 This is a thin layer over librdf, but should should also account for redland and others. Missing is constructing a graph for use with Graphpath (or some way of evaluating paths against the store), adding triples, and contexts (which may be useful for realoading things, for a start). ] [Swap stylesheet for template Michael Bridgen **20060103233757 The template classes were a little muddled; I've changed to the idea of a stylesheet, which (possibly) will be a kind of template, but for now standalone. My speculative superclasses for templates aren't much use, now I have a concrete and more useful set of classes. ] [Filesystem source stores entries as members of the feed Michael Bridgen **20060105004548 This includes a slightly ugly hack to be able to set an update time for the feed --- I return both the name and the timestamp of entries when loading them. Better would be .. ? ] [Fixing paths and (abstract) stylesheets Michael Bridgen **20060105004932 I moved a bunch of things into stylesheet from xmlstylesheet; some almost concrete classes that are independent of syntax. ] [Since the store commits to using rdflib, may as well import the adapter. Michael Bridgen **20060105005131] [A simple parser for a CSS-alike syntax for stylesheets. Michael Bridgen **20060105005516] [A regular expression step for graphpath. Michael Bridgen **20060105024216 Works for common usage I think: Class(atom.entry)[Property(atom.title)/RegExp(wiki_re)] ] [Use datetimes for updated values, instead of mtimes Michael Bridgen **20060105190103] [More examples, and some messy notes on stuff to do and figure out Michael Bridgen **20060105190144] [Don't include the filesystem root in URLs Michael Bridgen **20060105234907] [Add (incomplete but working) feedparser source Michael Bridgen **20060106024507] [Better values for fssource's nodes and properties Michael Bridgen **20060106025247 It's probably better to use the atom.id as the feed/entry resource. Since rdflib converts to strings (unicode strings) anyway, best to format our updated timestamp correctly too. ] [Factor store_* procedures into source.py and make fssource use them Michael Bridgen **20060108174101 There's still some knots; mostly about what to use as IDs for feeds, entries, and tags. Using the tag: URL scheme seems a good idea for now. ] [Assorted, indiscriminate patches for R1 Michael Bridgen **20060108174725 I've just committed the whole lot at once, since I'm not that concerned what they are (just that darcs stops telling me about them). Mostly just playing around; it's possible that the repo prior to this patch was more consistent. ] [Sketch of config and a CGI script, plus a stylesheet Michael Bridgen **20060108211333 An example config, a minimal CGI script (no headers). First example of a stylesheet. A modification to the stylesheet processor, to account for using blank nodes with values and types --- this should be extended to convert for various values. ] [Moving more R1 things out of the root directory Michael Bridgen **20060108211812] [Account for relative paths in stylesheets Michael Bridgen **20060108213831 This may be a bit of a hack, but will be mostly correct; better would be to evaluate the expression and find the place to put Current(). ] [Notes on permalinks and flavours Michael Bridgen **20060108234456] [Improving cgi script-ability Michael Bridgen **20060109005118 - Settled on consistent semantics for the internal format: use IDs for the feed and entry nodes, and give alternative links for flavours of them. Now finding the subject of a URL is just looking for the thing that has, as an alternate link, that URL (and MIME type). - Some more consistency in using atom.type instead of rdf.type, when the type is not an RDF class but an atom-specific attribute. - Added a pseudo-step 'Value' for stylesheets, to make it easier to do equality tests like Property(atom.type)/Value("text/html") ] [Allow Sort() to be told whether to sort ascending or descending, and provide operators < and > respectively Michael Bridgen **20060109005855] [Discriminate between match paths and select paths Michael Bridgen **20060109011527 Match paths (i.e., things to be tested for matching the current node) and select paths (things used to apply templates or output values) differ in where the implied 'current' node is; with a match node, the current node is what is being tested, while for select paths the current node is the implied starting point. ] [Use /> to end stylesheet ops, so sort operators can be used in paths Michael Bridgen **20060109030353] [Notes notes notes (look at the diffs) Michael Bridgen **20060109030528] [Use a buffer instead of print, so no extra linebreaks Michael Bridgen **20060109030617] [StylesheeApply now has a default template, which is to output the value of the selection as per StylesheetValue Michael Bridgen **20060109030658] [Account for possibly value conversions Michael Bridgen **20060110171932 The value operation now asks the stylesheet to find the value of a selection; this is for two purposes: firstly, to account for the encoding of things where a blank node with a property rdf:type is used; secondly, to allow the type of such a node to be checked, and if necessary, the actual value to be formatted for the output (whatever the stylesheet decides is appropriate). ] [A quick readme, just to record what libs I'm using for now. Michael Bridgen **20060114223936] [Stylesheet refactoring Michael Bridgen **20060114224249 Some better abstractions; Stylesheet and TemplateRule are the main ones, and stylesheetparser now has the implementations. Added conversions to the config; some commonality in dealing with flavours and types now becoming apparent. ] [Added simply so I have a reminder of using SAX Michael Bridgen **20060114224817 .. but I don't think I'll ever actually use this code, so I'll remove it again in a minute. ] [Remove XMLStylesheet again, it is horrible Michael Bridgen **20060114224849] [Update TODO notes to account for recent work; use alternate links for finding subject (and flavour) as suggested in TODO Michael Bridgen **20060114232020] [Only use the part of the file path below the root given for entry names Michael Bridgen **20060114233312] [Basic HTTP headers Michael Bridgen **20060115214729] [Supply flavours to fssource Michael Bridgen **20060115224755 I now deal with flavours more consistently. In feeds generated from the filesystem, they are represented in a list of pairs (flavour, url_generator), where url_generator takes the name of the file and provides the (link ref="alternate") href for it. Along with a few other changes to related procedures, this means that both the URL generation and ID generation are kept out of the feed generator. ] [Don't escape titles in sources, as they get escaped on output (if necessary) Michael Bridgen **20060116001052] [Stylesheet for an atom feed of a generate source Michael Bridgen **20060116001144] [PATH_INFO starts with a '/', so don't include it in the constructed request URL Michael Bridgen **20060116002445] [Update TODO to account for latest work, and new ideas Michael Bridgen **20060116003417] [Notes on root node matching in stylesheets Michael Bridgen **20060116132828] [Make the default (command-line) path look like a PATH_INFO Michael Bridgen **20060116133854] [Store external feeds as part of the uber-feed, with a source of the original feed (as per the notes in TODO) Michael Bridgen **20060119145330 Since we are aiming to aggregate feeds, what we are really constructing is one big feed; hence, when we add a feed to the store, we say it is in *our* feed, but use the original as the source of each entry. ] [Store, and display, author details for entries Michael Bridgen **20060119145644] [Improved handling of updated, which may not be present for entries; also, show /latest/ entries first in the HTML template Michael Bridgen **20060119145906] [Handle not being able to find a subject by showing a 404 Michael Bridgen **20060119150553] [Refactor stylesheet processing into class Michael Bridgen **20060120155119 A stylesheet is just a set of template rules. Since we may want to keep state while processing a stylesheet (for example, the root subject), I've factored the processing out into its own class: Processor. It is now the processor that encapsulates how paths are evaluated, and what values are; hence, it is now the processor that GraphpathProcessor (nee GraphpathStylesheet) subclasses, rather than Stylesheet (which no longer needs to be abstract). This move the method 'render' out of Stylesheet and gives Processor only 'render_subject'; so a convenience 'render' is provided in stylesheetparser. It might bear moving out of there into (default) config. ] [Rationalisation of stylesheet implementation Michael Bridgen **20060120160620 Gave th file a better name, and moved the unnecessary render procedure into config. ] [Root step, for matching the original subject Michael Bridgen **20060120171645 Root() is another placeholder, like Current(), that can be used in stylesheets to match the top-most subject. It operates a bit differently to XPath, in which there is a distinguished root, distinct from the document element. I'm not sure how to capture XPath-like semantics; I'd have to treat each kind of pattern involving Root() differently (maybe overriding the operators for Root is the way to do it). ] [Updated notes: aggregating done (more or less), root node match partially done Michael Bridgen **20060120172756] [Better handling of categories/tags Michael Bridgen **20060126175852] [TODO notes; base case is further off now :-S Michael Bridgen **20060126191723] [Tidy up config and give example of feed import Michael Bridgen **20060126191808] [A basic textile-based conversion Michael Bridgen **20060126191845] [Correct exception handling syntax Michael Bridgen **20060126222815] [Calendar support Michael Bridgen **20060128133235 Added support for describing calendar intervals, and a procedure for assigning calendar intervals to arbitrary subjects given a timestamp. It could do with some refactoring. Put code in the config to assign calendar dates to entries (but it is not used in the stylesheets yet). ] [Improved calendar support: calendar clock descriptions have inherent values now Michael Bridgen **20060130193041 It's convenient to have calendar clock descriptions (which are nicely selectable) have an rdf:value (which are nicely displayable). In the OWL-time spec, there is a DataTypeProperty which points at an xsd:datetime of the appropriate subtype; I'm shortcutting this by just having rdf:value give a string picture of the description. Also: display months in the archive (though not as links, since I've not implemented links for other than entries, which should be a plugin of some kind). ] [Fixed some defaults for stylesheets Michael Bridgen **20060130193524 1. Using a '/' at the start of a select pattern didn't work; 2. Node is useful in patterns after all, since some 'constants' are named in modules (owltime.unitMonth, say). 3. Values weren't quite being treated correctly; now deals better with values of blank nodes ] [Show authors, tags, and an archive Michael Bridgen **20060130200118 Mostly changes to the example stylesheet, but also a fix to fssource --- I forgot to include the author information. ] [Move implementation into a subdirectory, so less likely to interfere with system packages ('path', for example, walks the line, and 'calendar' would have been too much) Michael Bridgen **20060131114442] [Move calendar into a 'plugins' (really just optional things) directory Michael Bridgen **20060131120010] [Include entries in all calendar clock units, so can select them more easily Michael Bridgen **20060131200127 It's much easier to select entries by calendar-clock-description if each entry is inCalendarClock all of the relevant descriptions; i.e., the year, and month and day. I can't see any downside to this. Originally I was going to limit it to just day, and use int(erval)Contains relations between the descriptions. ] [A procedure for assigning permalinks to subjects Michael Bridgen **20060131200627 I wanted to be able to assign links to things other than stories, so that they would become addressable through the CGI script. Hence I've taken permalink generation out of fssource and made it a plugin procedure which can be run on any collection. It involves a bit more work in config, but it means I can then easily add them to calendar intervals and then be able to view those (with appropriate stylesheet rules). ] [Add calendar view to stylesheet, and other improvements Michael Bridgen **20060131201115] [Let calendar intervals be links (by default) Michael Bridgen **20060131201745] [Updated TODO Michael Bridgen **20060201141621] [Example story Michael Bridgen **20060203210030] [Give tags links as well, and order the archives Michael Bridgen **20060203210053 We can now refer to tags by URL as well; except there's a problem, which is that the stylesheet rules are called recursively since the tag is both the root and matched as part of an entry. ] [Tag stylesheet rule (danger! recursive) Michael Bridgen **20060203210300] [Another story on not having finished fuschia; handy htaccess example; notes on problem with duplicate blank nodes. Michael Bridgen **20060319230914] [Some fixes from moving things into plugins Michael Bridgen **20060319224143] [Notes on avoiding infinite recursion. Michael Bridgen **20060322142829] [Make sure the directory index file is set correctly. Michael Bridgen **20060322220852] [Keep track of an evaluation stack in the stylesheet processor. Michael Bridgen **20060322235446 As part of preventing recursion, we keep a stack of frames; these are also reflected in the graph. I tried originally to change the processor into a stack-machine, but because of the IO it's difficult to get everything to be output in order. Remaining problem: the predicate operator in Graphpath intersects with the LHS; so you can't have for example Class(atom.entry)[Context()/Node(rdf.nil)] and expect it to match anything. ] [Update notes on avoiding recursion: I need guards on templates, basically. Michael Bridgen **20060323122823] [Make the Sort repr depend on the sort order, as it should. Michael Bridgen **20060325164046] [Remove debug output --- ops Michael Bridgen **20060325164145] [Implement a Guard step, which is of immediate use in template match Michael Bridgen **20060325164343 expressions to stop infinite recursion. ] [Update TODO notes after fixing the big problem with recursive templates. Michael Bridgen **20060325165626] [Avoid duplicate tags and calendar entries. Michael Bridgen **20060325174615 For tags (and persons), it now checks to see if there is an identical node present, and returns that if so. For calendar entries, I'd forgotten to tell it to update the last date it compares with to see if it should make new nodes. ] [Pop from the stack properly, and give the opportunity to report on context exits Michael Bridgen **20060328224856] [Some sylesheet improvements; use a CSS stylesheet for a start Michael Bridgen **20060328225012] [Make the tags and months into list items, for extra semantic-ness and for extra styling-ness Michael Bridgen **20060328225635] [Push and pop to the end of the list rather than the start -- more idiomatic Python I think Michael Bridgen **20060328230342] [Include summaries, and check for existence of content Michael Bridgen **20060329124241] [Notes on intepreting feeds, dates, and plugins for specific feeds. Michael Bridgen **20060329203828] [An optional transform for the feedparser; could be cleaner but will do for now Michael Bridgen **20060330063953] [Encode the output as UTF8; the default appears to be ASCII, which can bork with some characters Michael Bridgen **20060330064109] [Notes on tag links, dates. Michael Bridgen **20060330102710] [Use the parsed updated value if possible, for consistency Michael Bridgen **20060329202911] [Retain the sorted list after sorting it. I sometimes wonder. Michael Bridgen **20060403065058] [Update TODO Michael Bridgen **20060422100420 I've actually completed, or obviated, a few of these. Instead of adding features at the minute, I want to concentrate on making what's there easier and more polished. ] [Shorter syntax for stylesheets Michael Bridgen **20060422123356 - Discard '_' as a synonym for Any() - Allow URIRefs and values to appear without wrapping in Node. This is done with a few judicious operator overloads, some of which may be fragile as they reach into graphpath and rdflib. - Use unary '+' as a shorthand for 'Property(...)'. It's not the best notation, but it is recognisable and far less verbose. This required a bit of fooling around, since [] has greater precedence than unary +. ] [Some kind of style. Michael Bridgen **20060422180635] [CSS file to support claim of style Michael Bridgen **20060422180800] [Use a picture of an actual fuschia in the style .. Michael Bridgen **20060423165700] [More notes on stylesheet language, and improvements to the stylesheet Michael Bridgen **20060423191734] [Story on problems with recursive templates Michael Bridgen **20060423193009] [Update atom stylesheet with new stylesheet syntax Michael Bridgen **20060423194143] [Give authors and contributors an rdf type Michael Bridgen **20060424080421 In the interest of being able to pick out things that refer to people, I've given author and contributor blank nodes those respective classes. That means atom.author and atom.contributor are used both as properties and classes; the wisdom of that remains to be seen. ] [TODO notes, mainly on paths and data input Michael Bridgen **20060702140415] [Clean up for release Michael Bridgen **20060702143144] [Allow for local configuration Michael Bridgen **20060702162701] [Allow xml:base to specify the base URL of the feed. Michael Bridgen **20060703222739 This makes it easier to supply a BASE element in the HTML templates, for example. store_feed now recognises the (feedparser-y) item 'xml_base' for feeds, though not yet for entries or anything else that could potentially have it. While doing this I noticed the signature of store_from_files shouldn't be storing the feed itself, so I moved that out into the config and fixed up the signature. The HTML stylesheet now has a BASE for relative URL resolution (like the stylesheet and images). ] [Fix default content dir. Michael Bridgen **20060703222935] [Added timetamps to stories and fixed typo. Michael Bridgen **20060703223255] [Adapt to work with later versions of rdflib Michael Bridgen **20060815164349 The main difference is the introduction of the Graph class. - Check the RDFLib version to see which procedures we should use - Move some of the namespaces around: fuschia.store is no longer responsible for having atom, rdf, rdfs (possibly they should all be put in one file) - Provide an adapter for the new RDFLib Graph class for graphpath ]