📚 Introduction

react-native-vision-camera-face-detector is a React Native library that integrates with the Vision Camera module to provide face detection functionality. It allows you to easily detect faces in real-time using device's front and back camera.

Is this package usefull to you?

Or give it a ⭐ on GitHub.

🏗️ Features

• Real-time face detection using front and back camera
• Adjustable face detection settings
• Optional native side face bounds, contour and landmarks auto scaling
• Can be combined with Skia Frame Processor

🧰 Installation

1yarn add react-native-vision-camera-face-detector

Then you need to add react-native-worklets-core plugin to babel.config.js. More details here.

🪲 Knowing Bugs

There are open issues (here, here, and here) about a bug on Skia Frame Processor that may cause a Black Screen on some Android Devices. This bug can be easily fixed with this trick but it makes Frame drawings to be in incorrect orientation.

💡 Usage

Recommended way (see Example App for Skia usage):

1import { 
2  StyleSheet, 
3  Text, 
4  View 
5} from 'react-native'
6import { 
7  useEffect, 
8  useState,
9  useRef
10} from 'react'
11import {
12  Frame,
13  useCameraDevice
14} from 'react-native-vision-camera'
15import {
16  Face,
17  Camera,
18  FaceDetectionOptions
19} from 'react-native-vision-camera-face-detector'
20
21export default function App() {
22  const faceDetectionOptions = useRef<FaceDetectionOptions>( {
23    // detection options
24  } ).current
25
26  const device = useCameraDevice('front')
27
28  useEffect(() => {
29    (async () => {
30      const status = await Camera.requestCameraPermission()
31      console.log({ status })
32    })()
33  }, [device])
34
35  function handleFacesDetection(
36    faces: Face[],
37    frame: Frame
38  ) { 
39    console.log(
40      'faces', faces.length,
41      'frame', frame.toString()
42    )
43  }
44
45  return (
46    <View style={{ flex: 1 }}>
47      {!!device? <Camera
48        style={StyleSheet.absoluteFill}
49        device={device}
50        faceDetectionCallback={ handleFacesDetection }
51        faceDetectionOptions={ faceDetectionOptions }
52      /> : <Text>
53        No Device
54      </Text>}
55    </View>
56  )
57}

Or use it following vision-camera docs:

1import { 
2  StyleSheet, 
3  Text, 
4  View 
5} from 'react-native'
6import { 
7  useEffect, 
8  useState,
9  useRef
10} from 'react'
11import {
12  Camera,
13  runAsync,
14  useCameraDevice,
15  useFrameProcessor
16} from 'react-native-vision-camera'
17import { 
18  Face,
19  useFaceDetector,
20  FaceDetectionOptions
21} from 'react-native-vision-camera-face-detector'
22import { Worklets } from 'react-native-worklets-core'
23
24export default function App() {
25  const faceDetectionOptions = useRef<FaceDetectionOptions>( {
26    // detection options
27  } ).current
28
29  const device = useCameraDevice('front')
30  const { detectFaces } = useFaceDetector( faceDetectionOptions )
31
32  useEffect(() => {
33    (async () => {
34      const status = await Camera.requestCameraPermission()
35      console.log({ status })
36    })()
37  }, [device])
38
39  const handleDetectedFaces = Worklets.createRunOnJS( (
40    faces: Face[]
41  ) => { 
42    console.log( 'faces detected', faces )
43  })
44
45  const frameProcessor = useFrameProcessor((frame) => {
46    'worklet'
47    runAsync(frame, () => {
48      'worklet'
49      const faces = detectFaces(frame)
50      // ... chain some asynchronous frame processor
51      // ... do something asynchronously with frame
52      handleDetectedFaces(faces)
53    })
54    // ... chain frame processors
55    // ... do something with frame
56  }, [handleDetectedFaces])
57
58  return (
59    <View style={{ flex: 1 }}>
60      {!!device? <Camera
61        style={StyleSheet.absoluteFill}
62        device={device}
63        isActive={true}
64        frameProcessor={frameProcessor}
65      /> : <Text>
66        No Device
67      </Text>}
68    </View>
69  )
70}

As face detection is a heavy process you should run it in an asynchronous thread so it can be finished without blocking your camera preview. You should read vision-camera docs about this feature.

Face Detection Options

Option	Description	Default	Options
performanceMode	Favor speed or accuracy when detecting faces.	fast	fast, accurate
landmarkMode	Whether to attempt to identify facial landmarks: eyes, ears, nose, cheeks, mouth, and so on.	none	none, all
contourMode	Whether to detect the contours of facial features. Contours are detected for only the most prominent face in an image.	none	none, all
classificationMode	Whether or not to classify faces into categories such as 'smiling', and 'eyes open'.	none	none, all
minFaceSize	Sets the smallest desired face size, expressed as the ratio of the width of the head to width of the image.	0.15	number
trackingEnabled	Whether or not to assign faces an ID, which can be used to track faces across images. Note that when contour detection is enabled, only one face is detected, so face tracking doesn't produce useful results. For this reason, and to improve detection speed, don't enable both contour detection and face tracking.	false	boolean
autoMode	Should handle auto scale (face bounds, contour and landmarks) and rotation on native side? If this option is disabled all detection results will be relative to frame coordinates, not to screen/preview. You shouldn't use this option if you want to draw on screen using Skia Frame Processor. See this and this for more details.	false	boolean
windowWidth	* Required if you want to use autoMode. You must handle your own logic to get screen sizes, with or without statusbar size, etc...	1.0	number
windowHeight	* Required if you want to use autoMode. You must handle your own logic to get screen sizes, with or without statusbar size, etc...	1.0	number
cameraFacing	Current active camera	front	front, back

🔧 Troubleshooting

Here is a common issue when trying to use this package and how you can try to fix it:

• Regular javascript function cannot be shared. Try decorating the function with the 'worklet' keyword...:
  • If you're using react-native-reanimated maybe you're missing this step.
• Execution failed for task ':react-native-vision-camera-face-detector:compileDebugKotlin'...:
  • This error is probably related to gradle cache. Try this sollution first.
  • Also check this comment.

If you find other errors while using this package you're wellcome to open a new issue or create a PR with the fix.

👷 Built With

• React Native
• Google MLKit
• Vision Camera

🔎 About

This package was tested using the following:

• react-native: 0.76.9 (new arch disabled)
• react-native-vision-camera: 4.6.4
• react-native-worklets-core: 1.5.0
• @shopify/react-native-skia: 1.5.0
• react-native-reanimated: ~3.16.1
• @react-native-firebase: ^22.2.0
• expo: ^52

Min O.S version:

• Android: SDK 26 (Android 8)
• IOS: 15.5

Make sure to follow tested versions and your device is using the minimum O.S version before opening issues.

📚 Author

Made with ❤️ by luicfrr