This paper proposes a service-oriented architecture for video analysis which separates object detection from event recognition. Our aim is to introduce new tools to be considered in the pathway towards Cognitive Vision as a support for classical Computer Vision techniques that have been broadly used by the scientific community. In the article, we particularly focus in solving some of the reported scalability issues found in current Computer Vision approaches by introducing an experience based approximation based on the Set of Experience Knowledge Structure (SOEKS). In our proposal, object detection takes place client-side, while event recognition takes place server-side. In order to implement our approach, we introduce a novel architecture that aims at recognizing events defined by a user using production rules (a part of the SOEKS model) and the detections made by the client using their own algorithms for visual recognition. In order to test our methodology, we present a case study, showing the scalability enhancements provided.