File and Object Storage Solutions for Big Data

Modern work environments heavily depend on content delivery and associated storage. Distributed and geographically dispersed consumption of research data, reports and documentation are rapidly becoming a huge challenge for traditional provisioning and distribution methods. Even a few years ago, repositories with billions of file objects and petabytes of storage space where unheard of.

High growth, but low change rates and long preservation times prevail in such content management systems. Traditional file system paradigms are stressed not only by growth but also by lack of transparent replication and distribution mechanisms.

Delivery paths have changed from accessing file systems - - being replaced by web delivery, metadata directories, search engines and associated access methods.

A new web-based paradigm based on no-filesystem object storage can solve the problems created by the traditional file-system approach. Web Object data repositories represent a flat name space that can be randomly accessed by an object identifier, provided by any application handling the descriptive metadata of an object.

Geographically dispersed data management systems are important for fast and low latency access to data but also for disaster recovery and continuity of data service. Object storage covers the traditional file system approach, but in addition offers more sophisticated information access methods, which also can change over the lifetime of a given dataset.

This presentation talks about the possibilities of a replicated, multi site single namespace web object storage solution and discusses use cases and deployment scenarios.