support - svm visualization
Plotting data from an svm fit-hyperplane (2)
I used svm to find a hyperplane best fit regression dependent on q, where I have 4 dimensions: x, y, z, q.
fit <- svm(q ~ ., data=data,kernel='linear')
and here is my fit object:
Call: svm(formula = q ~ ., data = data, kernel = "linear") Parameters: SVM-Type: C-classification SVM-Kernel: linear cost: 1 gamma: 0.3333333 Number of Support Vectors: 1800
I have a 3d plot of my data, where the 4th dimension is color, using plot3d. How can I overlay the hyperplane that svm found? How can I plot the hyperplane? I'd like to visualize the regress hyperplane.
I'm just starting out in R myself, but there's a decent tutorial on using the e1071 package in R for regression rather than classification:
with a zip file of the test dataset and R script in:
Skip the first section on Tanagra and head straight to section 6 (page 14). It has its faults, but it gives examples of using R for linear regression, SVR with epsilon-regression and with nu-regression. It also makes a stab at demonstrating the tune() method (but could be done better, IMHO).
(Note: if you choose to run the examples in that paper, don't bother trying to find a working copy of xlsReadWrite -- it's much easier to export qsar.xls as a .csv file and just use read.csv() to load the dataset.)
I used svm to find a hyperplane best fit regression
But according to:
Call: svm(formula = q ~ ., data = data, kernel = "linear") Parameters: SVM-Type: C-classification
you are doing classification.
So, first of all decide what you need: to classify or to fit regression, from
?svm, we see:
type: ‘svm’ can be used as a classification machine, as a regression machine, or for novelty detection. Depending of whether ‘y’ is a factor or not, the default setting for ‘type’ is ‘C-classification’ or ‘eps-regression’, respectively, but may be overwritten by setting an explicit value.
As I believe you didn't change the parameter
type from its default value, you are probably solving
classification, so, I will show how to visualize this for classification.
Let's assume there are
2 classes, generate some data:
> require(e1071) # for svm() > require(rgl) # for 3d graphics. > set.seed(12345) > seed <- .Random.seed > t <- data.frame(x=runif(100), y=runif(100), z=runif(100), cl=NA) > t$cl <- 2 * t$x + 3 * t$y - 5 * t$z > t$cl <- as.factor(ifelse(t$cl>0,1,-1)) > t[1:4,] x y z cl 1 0.7209039 0.2944654 0.5885923 -1 2 0.8757732 0.6172537 0.8925918 -1 3 0.7609823 0.9742741 0.1237949 1 4 0.8861246 0.6182120 0.5133090 1
Since you want
kernel='linear' the boundary must be
w1*x + w2*y + w3*z - w0 - hyperplane.
Our task divides to 2 subtasks: 1) to evaluate equation of this boundary plane 2) draw this plane.
1) Evaluating the equation of boundary plane
First, let's run
> svm_model <- svm(cl~x+y+z, t, type='C-classification', kernel='linear',scale=FALSE)
I wrote here explicitly
type=C-classification just for emphasis we want do classification.
scale=FALSE means that we want
svm() to run directly with provided data without scaling data (as it does by default). I did it for future evaluations that become simpler.
svm_model doesn't store the equation of boundary plane (or just, normal vector of it), so we must evaluate it. From svm-algorithm we know that we can evaluate such weights with following formula:
w <- t(svm_model$coefs) %*% svm_model$SV
The negative intercept is stored in
svm_model, and accessed via
2) Drawing plane.
I didn't find any helpful function
plane3d, so, again we should do some handy work. We just take grid of pairs
(x,y) and evaluate the appropriate value of
z of the boundary plane.
detalization <- 100 grid <- expand.grid(seq(from=min(t$x),to=max(t$x),length.out=detalization), seq(from=min(t$y),to=max(t$y),length.out=detalization)) z <- (svm_model$rho- w[1,1]*grid[,1] - w[1,2]*grid[,2]) / w[1,3] plot3d(grid[,1],grid[,2],z) # this will draw plane. # adding of points to the graphics. points3d(t$x[which(t$cl==-1)], t$y[which(t$cl==-1)], t$z[which(t$cl==-1)], col='red') points3d(t$x[which(t$cl==1)], t$y[which(t$cl==1)], t$z[which(t$cl==1)], col='blue')
We did it with
rgl package, you can rotate this image and enjoy it :)