A set of standard tools has to be selected among a wide set of possibilities. What type of database to choose for storing and retrieving Metadata efficiently and scale with large amounts of data?
- Free and open source and running under Windows and Linux.
- Capable of handling small binary data (the big stuff is stored on the filesystem) but many features require small amounts of binary data to be stored in a database.
- Fast & scalable enough
- .NET interface
Various non RDBMS systems excluded are the Graph DBs that build a class on their own in my opinion.
[http://wiki.github.com/tinkerpop/gremlin Gremlin] is a kind of ORM for Graph DBs. Gargamel also seems of interest.
The NoSQL tools allow a great scalability and easy synchronisation (replication) that can be used also for continuous backup purposes.
A MongoDB and CakePHP solution is currently used for the proof of concept.
A collection of the most interesting links I found on the topic.
[http://techportal.ibuildings.com/2009/09/07/graphs-in-the-database-sql-meets-social-networks Amazing article about storing trees in SQL databases]
[http://blog.boxedice.com/2009/07/25/choosing-a-non-relational-database-why-we-migrated-from-mysql-to-mongodb Choosing a non relational database]
[http://bjclark.me/2009/08/04/nosql-if-only-it-was-that-easy NoSQL if only it were that easy]
[http://jayant7k.blogspot.com/2009/08/document-oriented-data-stores.html Comparison Document Oriented Data Stores]
[http://poorbuthappy.com/ease/archives/2007/04/29/3616/the-top-10-presentation-on-scaling-websites-twitter-flickr-bloglines-vox-and-more Scaling websites] (I really liked the 14 rules in the Yahoo presentation)
HBase Vs Cassandra
- [http://www.roadtofailure.com/2009/10/29/hbase-vs-cassandra-nosql-battle http://www.roadtofailure.com/2009/10/29/hbase-vs-cassandra-nosql-battle]
HBase Vs CouchDB
- [http://www.larsgeorge.com/2009/03/hbase-vs-couchdb-in-berlin.html http://www.larsgeorge.com/2009/03/hbase-vs-couchdb-in-berlin.html]
MongoDB Vs CouchDB
- [http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database On building a stupidly fast graph database]
Architecture of famous websites
[http://www.microsoft.com/casestudies/CaseStudyDetail.aspx?casestudyid=4000004532 MySpace case study]
By using a NoSQL solution there is no need of an ORM.
The only ones I know well are NHibernate and Castle ActiveRecord. Therefore I would tend to use them. But it may not be the best solution. I removed the non open source tools.
ADO.NET Entity Framework, Microsoft's ORM (released with .NET 3.5 SP1)
Language Integrated Query#LINQtoSQL|LINQ to SQL, Free, .Net framework component
SubSonic, free ORM and code generation tool backed by Microsoft
- NHibernate, open source
- Castle ActiveRecord, ActiveRecord for .NET, open source
- Fluent NHibernate, open source and free
- DataObjects.Net, open source, commercial
- Atlas (software)|Atlas,open source
- Business Logic Toolkit for .NET, open source
- Crystal Mapper, open source
- Developer Express, eXpress Persistent Objects (XPO)
- Euss, open source
- Habanero.NET|Habanero, Free open source Enterprise application framework with a Free Code Generation Tool
- iBATIS, Free open source
- Neo (object-relational toolset)|Neo, open source
- ObjectMapper .NET, GPL and commercial license
- Picasso (software)|Picasso, Open-Source ORM Framework & Code Generator (Relational & Xml), free with Commercial support available
- Telerik OpenAccess ORM|OpenAccess, free or commercial
- TierDeveloper, free ORM and code generation tool
- Sooda, open source; BSD license
- Subsonic (software)|Subsonic, open source
- Logic Data Access, open source