In recent years, the volume and the complexity of data in Building Automation System networks have increased exponentially. As a result, a manual analysis of network traffic data has become nearly impossible. Even automated but supervised methods are problematic in practice since the large amount of data makes manual labeling, required to train the algorithms to differentiate between normal traffic and anomalies, impractical. This paper introduces a framework which allows the characterization of BACnet network traffic data by means of unsupervised machine learning techniques. Specifically, we use clustering, random forests, one-class support vector machines and support vector classifier, after a pre-processing step that includes principal components analysis for dimensionality reduction. We compare the effectiveness of the methods in detecting anomalies by performing experiments on BACnet network traffic data from various sources. We describe which of these unsupervised methods work best in specific scenarios since each method has its distinct advantages and disadvantages. In particular, we discuss which method is best suited to detect new types of anomalies (novelty detection), or which method most reliably and efficiently finds new attacks of a type that has been captured in the data previously.