-
Notifications
You must be signed in to change notification settings - Fork 0
Recognizer
Recognizer class allow the developpeur to recognize a face of someone who already have enregistred is face in a folder in a picture or a video. With the class you can also add image to the current recognizer environment and save the image for recognize the people next time. All the face image stored is associated with a label (int) and a name (string). The faceRecognizer class inside the project is copied on face module in opencv_contrib. I have chose to use it without all opencv_contrib for better portability and less weight. If you want more information on it because I don't explain this so much below please see : facerec_api.html
- Recognizer(int widht = 92, int heihgt = 112);
Initialize a new instance of Recognizer class. Set the heigh and the width that the image in the current trainning will use. 92 and 112 are the value used by the ATDatabase.
width: Width of the image used in the training espace.
height: Height of the image used in the current training.
- void addFrameToCurrentTraining(cv::Mat frame, int label, std::string faceName);
This function is the first step for recognition. It will add the frame passed in parameter to the current training space for adding it for the next model training and the next recognition.
frame: Frame that contain a face that you want to add to the current training.
label: Label associated to the name and the face.
faceName: Name of the person who owns the face.
- void addFrameToCurrentTrainingAndSave(cv::Mat frame, int label, std::string faceName, std::string fileName, std::string folderName = "face/");
This function is the first step for recognition and for building a important database of face. It will add the frame passed in parameter to the current training space for adding it for the next model training and the next recognition and it will save the frame like an image in the specified folder with the specified name in the parameters.
frame: Frame that contain a face that you want to add to the current training.
label: Label associated to the name and the face.
faceName: Name of the person who owns the face.
fileName: Name given to the registered image.
folderName: Name of the folder in wich you want to save your face database.
- void askForAddImageToCurrentTrainingAndSave(cv::Mat noRecognizedFace, std::string folderName = "face/");
Usefull function that ask the user if he want to add the frame in parameter to the current training and the database and if the answer is yes it will call addFrameToCurrentTrainingAndSave().
noRecognizedFace: Image of a face that receive a too hight accuracy in the recognition process and that you want to ask the user if he want to add it in the database and the current training.
folderName: Name of the folder in wich you want to save your face if the user is okay.
- double recognize(cv::Mat faceToRecognize);
Main function of the class. Recognize() will print the name and the label of the person who looks the most like the picture passed in parameter. The function return the confidence of the recognize process. If he is high you can call askForAddImageToCurrentTrainingAndSave() for add a new people to the database.
faceToRecognize: Frame that contains a detected face we want to try to recognize.
- void train();
Train the current model. That is used for transform image in data associate to a label for the next comparaison with an image.
- void readCsv(const std::string& filename, char separator = ';');
Load the data into a CSV into the vector of the current training space. Image are stocked like path. When we read the CSV file we will read line by line the data. Anf for each line take all the character until ";". For the first string it's a path so we load the image at this path and add it to the current vector for training. We do the same for the second (label) and the third (name) but directly.
fileName: The name of the file we want to read.
separator: The special character used like separator of data in the csv.
- void saveCsv(std::string fileName = "customFaceCsv.txt");
Take the value of the path of the frame, the label and the name associated to a frame stored in the current training vector and save them in a single line separated with ";".
fileName: The name of the file in wich we want to save the data.
- void saveImg(std::string folderName, std::string nameOfImage, cv::Mat faceToSave);
Save a frame passed in parameter to the specified folder in the IMG_DIR defined in Constantes.h with the name passed in parameter. If the folder doesn't exist it will be created.
folderName: Name of the folder in wich we want to save the image.
nameOfImage: Name given to the registered image.
faceToSave: Image to save.
- cv::Mat processFrame(cv::Mat frameToProcess);
Rezize and change the color of a frame if needed for use regularized variables for the comparaison of two images.
frameToProcess: The original image that we want to regularize.
- bool isTrained();
Return true if the faceRecognizer model is training. While faceRecognizer model train we can't use others function of the model like predict().
- bool askForAddImageInProcess();
Return true if we are asking to the user if he want to add an image to the database. Usefull only if you want to execute askForAddImageToCurrentTrainingAndSave() in a separated thread.
- int getFrameWidth();
Return the width of the image used in our training database. Usefull if you want to check the compatibility between two image.
- int getFrameHeight();
Return the height of the image used in our training database. Usefull if you want to check the compatibility between two image.
- cv::Size getFrameSize();
Return the Size class of the image used in our training database. Usefull for cv::rezize() function. Remplace getFrameWidth() + getFrameHeight() in one function.
- double getLastConfidence();
Return the last confidence of _model->predict() function. Used to know if the name returned by recognize() function is the good one or if maybe there are an error.
- int getLabelFrameSize();
Return the size of the _labelFrame vector. Usefull to know if the adding frame to training space process worked fine.
- int getTrainingFramesSize();
Return the size of the _trainingFrame vector. Usefull to know if the adding frame to training space process worked fine.
- int getNumberOfFaceSauvegarded();
Need to be done. Get the number of different label in _labelFrame
- void printConf();
Print in the console all the data in the different vector used in the training space. Usefull to debug small database.
- cv::Ptrcv::face::FaceRecognizer getModel();
Return the model used in faceRecognizer class. Usefull if you need to call directly some function like train or predict.
- bool imageExist(const std::string name, const std::string folderName = "face/");
Return true if an image exist to the specifed path into IMG_DIR (Constantes.h) false otherwise.
name: Name of the image we want to check the existence.
folderName: The name of the folder that contains the image.
- void createDirectory(std::string folderName);
Create a directory in IMG_DIR (Constantes.h) for create a new database. Function used in windows and linux.
folderName: The name of the folder we want to create.
/*###############################################################################################################################
# #
# Sample for face recognition #
# #
#################################################################################################################################*/
int main(int argc, const char** argv)
{
//----------------------------------- variables initialisation ---------------------------------------------------------
std::thread t[2];
// Video is the class for use video and detection
Video myVideo(0); //0 is the defaut camera, use -1 for any cammera or a number between 1 to 99
Recognizer myRecognizer;
std::vector<cv::Mat> lastFacesDetected;
bool done = false;
int i = 0;
std::string namePersonRecognized, goodName;
time_t timer;
time(&timer); //set timer to the current date in ms in
cv::Mat imgNull;
//Launch video in a thread
t[0] = std::thread([&] {myVideo.start(); });
myVideo.startFaceDetect();
// ---------------------------------- first option load a model manually with files -----------------------------------
//cv::Mat person1 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person1.jpeg", 0);
//cv::Mat person2 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person2.jpeg", 0);
//cv::Mat person3 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person3.jpeg", 0);
//cv::Mat person4 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person4.jpeg", 0);
//cv::Mat person5 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person5.jpeg", 0);
//myRecognizer.addFrameToCurrentTraining(person1,1,"person1");
//myRecognizer.addFrameToCurrentTraining(person2, 2, "person2");
//myRecognizer.addFrameToCurrentTraining(person3, 3, "person3");
//myRecognizer.addFrameToCurrentTraining(person4, 4, "person4");
//myRecognizer.addFrameToCurrentTraining(person5, 5, "person5");
// ------------------------------- second option load a model automatically with csv file --------------------------------
try {
myRecognizer.readCsv(CSV_FACE_RECO); // see Constantes.h
}
catch (cv::Exception& e) {
std::cerr << "Error opening file \"" << CSV_FACE_RECO << "\". Reason: " << e.msg << std::endl;
getchar();
exit(1);
}
//-------------------------------------- Train the loaded data for face recognition -------------------------------------
t[1] = std::thread([&] {myRecognizer.train(); });
//If you want informations on the model used, uncomment these lines
//std::cout << "width = " << myRecognizer.getFrameWidth() << " height = " << myRecognizer.getFrameHeight() << std::endl;
//std::cout << "trainingFrame size = " << myRecognizer.getTrainingFramesSize() << " labelFrameSize = " << myRecognizer.getLabelFrameSize() << std::endl;
//myRecognizer.printConf();
//------------------------------------------------Recognize face in video ------------------------------------------
while (true)
{
//dont recognize if the model is not trained or if no face never get Recognize
if (abs(time(NULL) - timer) > 1 && !myRecognizer.isTrained() && !myVideo.getLastFacesDetected().empty() && !myRecognizer.equalTest(myVideo.getLastFacesDetected(), lastFacesDetected))//Maybe add myVideo.getLastFacesDetected() != oldLastFacesDetected for not recognize the same image again and again
{
lastFacesDetected = myVideo.getLastFacesDetected(); // get all the current face in video stream
i = 0;
myVideo.clearLabel(); // remove the old name associed to a face
for (std::vector<cv::Mat>::iterator face = lastFacesDetected.begin(); face != lastFacesDetected.end(); face++, i++)//for each face detected
{
namePersonRecognized = myRecognizer.recognize(*face);//launch the recognition
myVideo.addLabel(namePersonRecognized, i); //display the name on the video stream. If you only want to display him if the reognition is sur move this line under, in the else statement.
myVideo.setImgToPrint(*face); // display in an another windows in the video thread the region of interest corresponding to the face detected
if (myRecognizer.getLastConfidence() > 3000 && !myRecognizer.askForAddImageInProcess())//If we are not sur of the recognition and we are not already asking for add image if we ask in an other thread
{
myRecognizer.askForAddImageToCurrentTrainingAndSave(*face);//Function ask the user if he want to add his face to the database and do it if the answer is positif
//t[2] = std::thread([&] {myRecognizer.askForAddImageToCurrentTrainingAndSave(*face); }); // if you want to use multi threading but you need to use this differently to use when you implement your GUI (maybe a other windows specially created fot that or the question under the face displayed for informations)
}
else
{
if (!myRecognizer.askForAddImageInProcess()) //if we are not already asking for add image if we ask in an other thread
{
std::cout << "Good Name ? y/n" << std::endl; //Ask the user if the name given to the face detected is correct
std::cin >> goodName;
if (goodName == "n" || goodName == "N")//If the user said that the name isn't correct ask him if he want to add the correct name to the face in the database
{
myRecognizer.askForAddImageToCurrentTrainingAndSave(*face);
}
}
}
myVideo.setImgToPrint(imgNull);
}
//std::cout << "face analyzed = " << i << std::endl;
time(&timer);
}
cv::waitKey(1);
}
return 0;
}
You need to create your own database of face that you want to recognize. For that you can load image manually and save them or turn the video on and save image. Code:
/*###############################################################################################################################
# #
# Sample for create custom face database #
# #
#################################################################################################################################*/
int main(int argc, const char** argv)
{
//----------------------------------- variables initialisation ---------------------------------------------------------
std::thread t[2];
// Video is the class for use video and detection
Video myVideo(0); //0 is the defaut camera, use -1 for any cammera or a number between 1 to 99
Recognizer myRecognizer;
std::vector<cv::Mat> lastFacesDetected;
bool done = false;
int i = 0;
std::string namePersonRecognized, goodName;
time_t timer;
time(&timer); //set timer to the current date in ms in
cv::Mat imgNull;
//Launch video in a thread
t[0] = std::thread([&] {myVideo.start(); });
myVideo.startFaceDetect();
// ---------------------------------- first option load a model manually with files -----------------------------------
//cv::Mat person1 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person1.jpeg", 0);
//cv::Mat person2 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person2.jpeg", 0);
//cv::Mat person3 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person3.jpeg", 0);
//cv::Mat person4 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person4.jpeg", 0);
//cv::Mat person5 = cv::imread("C:/dev/git/PerceptLibrary/data/dataForRec/face/person5.jpeg", 0);
//myRecognizer.addFrameToCurrentTrainingAndSave(person1,1,"person1", "person1", "customDatabase");
//myRecognizer.addFrameToCurrentTrainingAndSave(person2, 2, "person2", "person1", "customDatabase");
//myRecognizer.addFrameToCurrentTrainingAndSave(person3, 3, "person3", "person1", "customDatabase");
//myRecognizer.addFrameToCurrentTrainingAndSave(person4, 4, "person4", "person1", "customDatabase");
//myRecognizer.addFrameToCurrentTrainingAndSave(person5, 5, "person5", "person1", "customDatabase");
// ------------------------------- second option load a model automatically with csv file --------------------------------
while (true)
{
//dont recognize if the model is not trained or if no face never get Recognize
if (abs(time(NULL) - timer) > 1 && !myVideo.getLastFacesDetected().empty() && !myRecognizer.equalTest(myVideo.getLastFacesDetected(), lastFacesDetected))//Maybe add myVideo.getLastFacesDetected() != oldLastFacesDetected for not recognize the same image again and again
{
lastFacesDetected = myVideo.getLastFacesDetected(); // get all the current face in video stream
i = 0;
for (std::vector<cv::Mat>::iterator face = lastFacesDetected.begin(); face != lastFacesDetected.end(); face++, i++)//for each face detected
{
myVideo.setImgToPrint(*face); // display in an another windows in the video thread the region of interest corresponding to the face detected
myRecognizer.askForAddImageToCurrentTrainingAndSave(*face, "customDatabase/");
myVideo.setImgToPrint(imgNull);
}
//std::cout << "face analyzed = " << i << std::endl;
time(&timer);
}
cv::waitKey(1);
}
return 0;
}
Change the recognition algorithm: In Recognizer.h : cv::Ptrcv::face::FaceRecognizer _model = cv::face::createEigenFaceRecognizer();
You can change createEigenFaceRecognizer() with createFisherFaceRecognizer() or createLBPHFaceRecognizer(). This change will be change the algorithm behind the face recognition and maybe one will correspond better at your need. For more information please see : http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html
If you want to use createFisherFaceRecognizer() you will have an error on the training function because of the presence to the same label for several image. For fix it just give only one label per image. (go to Recognizer::askForAddImageToCurrentTrainingAndSave and modify all the
addFrameToCurrentTrainingAndSave(noRecognizedFace, label[0], answer, fileName, folderName);
by
addFrameToCurrentTrainingAndSave(noRecognizedFace, getLabelFrameSize() + 1, answer, fileName, folderName);
Make sur that in the Csv file you read (if you read one) there is not 2 time the same label.
You can change the frameSize used by the training model for adpat it to your camera. If you want to do that take a picture with your camera call the function cv::Mat::cols and cv::Mat::rows store them into variables and print them.
You just need to change the value given in the Recognizer initialisation with your new values:
Recognizer myRecognizer(image.cols, image.rows);