aSVM – Adaptive Support Vector Machines

by Jun Yang @ CMU, with Rong Yan and Alex Hauptmann

Introduction

Adaptive Support Vector Machines (aSVM) is a software package implementing an algorithm for adapting one or multiple existing SVM (source) classifiers to a SVM (target) classifier for a new domain. For example, if there are two data domains as A and B with similar distributions, one can use aSVM to adapt a classifier on A to a classifier for B based on a small number of labeled data in B. The adapted classifier is expected perform better than a classifier trained only from the same amount of labeled examples. This is implemented based on the LIBSVM package.

The goal of aSVM is to improve classification on a new domain by leveraging the existing models from related domains. It has many real-world applications. For example, if you have a SVM model for classifying indoor/outdoor images trained from Corel collection, you can adapt it to classify indoor/outdoor photos in Flickr collection by labeling only a small number of photos. This paper contains some results of using this algorithm for classifying video scenes from different sources.

This package has the following features:

- adapt one or multiple source models

- average, manual, or automatic weighting of source model(s)

- accept standard data format (same as LIBSVM)

- output standard SVM model file (same as LIBSVM)

Download

the executable (only for Windows)

the source code in C++ (as Visual C++ project)

Use Guide

adapt_svm_train [options] [+ source_model_file[,weight]] train_data_file [output_model_file]

Options:

-t (default 2): Type of kernel function; 0 for linear, 1 for polynomial, 2 for radial basis function (RBF), 3 for sigmoid

-d (default 3): Degree in polynomial kernel function

-g (default 1/k, k as feature dimension): Gamma parameter in RBF kernel function

-c (default 1): Cost parameter C on classification error

-m (default 100): Cache memory size in MB

-e (default 0.001): Tolerance (epsilon in SVM) of termination criterion

-h (default 1): whether to use shrinking heuristics, 0 or 1

-wi (default 1): set the parameter C of class i to weight*C (e.g., -w1 1 –w2 1)

-l (default 0): whether to learn the weights of source models, 0 or 1

-s (default 1): Contribution factor of source models, effective only when –l 1

Source models:

You need to provide at least one or multiple source SVM model files (e.g., + model1.txt + model2.txt), which have to follow LIBSVM’s model file format. Note the space between + and the filename. You can optionally assign the weight of the model at the end of filename separated by comma (e..g, + model1.txt,0.8). If the weight is not specified, it is assumed to be 1. You can also choose to have these weights learned automatically by specifying “–l 1”. In this case, manually assigned weights are ignored.

Training data file:

The training data file contains the labeled examples from the target domain. The file must follow the format of LIBSVM’s data file, which looks like:

Each row represents one data point, where <label> is the label (-1 or 1), and <ID>:<value> pair denotes one of its feature components, with <feature> as the ID of the feature (i.e., order starting from 1). Zero-valued features can be omitted. Thus, each line can be of different length, depending on the number of non-zero features for each data point.

An example data file is given below, where each data point has a 10-d feature vector.

1 1:0.4 6:0.1 10:0.7

-1 3:0.2 7:0.15

-1 1:0.8 2:0.7 8:0.2 9:0.1

-1 2:0.1 4:0.3 7:0.7

Target model file:

You can specific the filename of the adapted model as the output of aSVM. If not specified, it will be named after the data file (i.e., <train_data_file>.model). This model file follows the same format as LIBSVM’s model file, which means you can run svm-predict.exe from LIBSVM on this model file.

Examples:

adapt_svm_train –t 2 –g 0.1 + model1.txt,0.8  + model2.txt,0.2 train.dat

or

adapt_svm_train –t 0 –l 1 –s 100 + model1.txt + model2.txt train.dat model.txt

Bug Reports

If you find bugs or you have problems with the code you cannot solve by yourself, please contact me via email juny@cs.cmu.edu.

Disclaimer

This software is free only for non-commercial use. It must not be modified and distributed without prior permission of the author. The author is not responsible for implications from the use of this software.

Reference of aSVM (please cite this if you find it useful)

Jun Yang, Rong Yan, Alex Hauptmann, Cross-Domain Video Concept Detection using Adaptive SVMs, ACM Multimedia 2007.

* Last modified May 26^th, 2008 by Jun Yang