JUCS - Journal of Universal Computer Science 21(13): 1810-1829, doi: 10.3217/jucs-021-13-1810
A Distributed Recommendation Platform for Big Data
expand article infoDaniel Valcarce, Javier Parapar§, Álvaro Barreiro
‡ University of A Coruña, A Coruña, Spain§ University of A Coruña, Coruña, Spain
Open Access
The vast amount of information that recommenders manage these days has reached a point where scalability has become a critical factor. In this work, we propose a scalable architecture designed for computing Collaborative Filtering recommendations in a Big Data scenario. In order to build a highly scalable and fault-tolerant platform, we employ fully distributed systems without any single point of failure. We study the use of data replication and data distribution technologies. Additionally, we consider different caching techniques. Taking into account these requirements, we propose particular technologies for each component of the platform. Next, we evaluate the response times of storing, generating and serving recommendations using MySQL Cluster and Cassandra showing that the latter technology is much more adequate for that purpose. Finally, we conduct a simulation for evaluating the impact of a memory caching system.
recommender systems, big data, scalability, architecture, NoSQL, cache