Using the Facemark API {#tutorial_facemark_usage}

Goals

In this tutorial will helps you to

Create a Facemark object.
Set a user defined face detector for the facemark algorithm
Train the algorithm.
Use the trained model to detect the facial landmarks from a given image.

Preparation

Before you continue with this tutorial, you should download the dataset of facial landmarks detection. We suggest you to download the helen dataset which can be retrieved at http://www.ifp.illinois.edu/~vuongle2/helen/ (Caution! The algorithm requires around 9GB of RAM to train on this dataset).

Make sure that the annotation format is supported by the API, the contents in annotation file should look like the following snippet: @code version: 1 n_points: 68 { 212.716603 499.771793 230.232816 566.290071 ... } @endcode

The next thing to do is to make 2 text files containing the list of image files and annotation files respectively. Make sure that the order or image and annotation in both files are matched. Furthermore, it is advised to use absolute path instead of relative path. Example to make the file list in Linux machine @code ls $PWD/trainset/.jpg > images_train.txt ls $PWD/trainset/.pts > annotation_train.txt @endcode

example of content in the images_train.txt @code /home/user/helen/trainset/100032540_1.jpg /home/user/helen/trainset/100040721_1.jpg /home/user/helen/trainset/100040721_2.jpg /home/user/helen/trainset/1002681492_1.jpg @endcode

example of content in the annotation_train.txt @code /home/user/helen/trainset/100032540_1.pts /home/user/helen/trainset/100040721_1.pts /home/user/helen/trainset/100040721_2.pts /home/user/helen/trainset/1002681492_1.pts @endcode

Creating the facemark object

@code /create the facemark instance/ FacemarkLBF::Params params; params.model_filename = "helen.model"; // the trained model will be saved using this filename Ptr facemark = FacemarkLBF::create(params); @endcode

Set a custom face detector function

Firstly, you need to create your own face detector function, you might also need to create a struct to save the custom parameter. Alternatively, you can just make these parameter hard coded within the myDetector function. @code struct Conf {

cv::String model_path;
double scaleFactor;
Conf(cv::String s, double d){
    model_path = s;
    scaleFactor = d;
    face_detector.load(model_path);
};

CascadeClassifier face_detector;

}; bool myDetector(InputArray image, OutputArray faces, Conf *conf){

Mat gray;

if (image.channels() > 1)
    cvtColor(image, gray, COLOR_BGR2GRAY);
else
    gray = image.getMat().clone();

equalizeHist(gray, gray);

std::vector<Rect> faces_;
conf->face_cascade.detectMultiScale(gray, faces_, conf->scaleFactor, 2, CASCADE_SCALE_IMAGE, Size(30, 30) );
Mat(faces_).copyTo(faces);
return true;

} @endcode

The following snippet demonstrates how to set the custom detector to the facemark object and use it to detect the faces. Keep in mind that some facemark object might use the face detector during the training process.

@code Conf config("../data/lbpcascade_frontalface.xml", 1.4); facemark->setFaceDetector(myDetector, &config); // we must guarantee proper lifetime of "config" object @endcode

Here is the snippet for detecting face using the user defined face detector function.

@code Mat img = imread("../data/himym3.jpg"); std::vectorcv::Rect faces; facemark->getFaces(img, faces, config); for(int j=0;j<faces.size();j++){

cv::rectangle(img, faces[j], cv::Scalar(255,0,255));

} imshow("result", img); waitKey(0); @endcode

Training a facemark object

First of all, you need to set the training parameters @code params.n_landmarks = 68; // number of landmark points params.initShape_n = 10; // number of multiplier for make data augmentation params.stages_n=5; // amount of refinement stages params.tree_n=6; // number of tree in the model for each landmark point params.tree_depth=5; //he depth of decision tree facemark = FacemarkLBF::create(params); @endcode
And then, you need to load the file list from the dataset that you have prepared. @code std::vector images_train; std::vector landmarks_train; loadDatasetList("images_train.txt","annotation_train.txt",images_train,landmarks_train); @endcode
The next step is to add training samples into the facemark object. @code Mat image; std::vector facial_points; for(size_t i=0;i<images_train.size();i++){
```
image = imread(images_train[i].c_str());
loadFacePoints(landmarks_train[i],facial_points);
facemark->addTrainingSample(image, facial_points);
```
} @endcode
execute the training process @code /train the Algorithm/ facemark->training(); @endcode

Use the trained model to detect the facial landmarks from a given image.

First of all, load the trained model. You can also download the pre-trained model in this link https://raw.githubusercontent.com/kurnianggoro/GSOC2017/master/data/lbfmodel.yaml @code facemark->loadModel(params.model_filename); @endcode
Detect the faces @code facemark->getFaces(img, faces, config); @endcode
Perform the fitting process @code std::vectorstd::vector<Point2f > landmarks; facemark->fit(img, faces, landmarks); @endcode
Display the result @code for(int j=0;j<faces.size();j++){ face::drawFacemarks(img, landmarks[j], Scalar(0,0,255)); } imshow("result", img); waitKey(0); @endcode

facemark_usage.markdown 5.5 KB History Raw