The complete EdgeVerify authentication pipeline is:
User
│
▼
Camera Preview
│
▼
Face Detection (ML Kit)
│
▼
Liveness Verification
(Blink → Smile → Left → Center → Right)
│
▼
Face Capture
│
▼
Face Crop
│
▼
Face Resize (112×112)
│
▼
MobileFaceNet (TensorFlow Lite)
│
▼
192-Dimensional Face Embedding
│
├── Registration
│ │
│ ▼
│ AsyncStorage
│
└── Verification
│
▼
Cosine Similarity
│
▼
Authentication Result
Purpose:
- Shows live camera preview
- Captures employee face image
- Provides pixel buffer to the AI pipeline
Key Technology:
- react-native-vision-camera
Why needed:
The face recognition model requires image data from the camera.
Purpose:
- Detects face position
- Provides bounding box coordinates
Output Example:
x = 203
y = 417
width = 371
height = 378
Key Technology:
- Google ML Kit Face Detection
Why needed:
The recognition model should only process the face region, not the entire image.
Purpose:
Prevent spoofing attacks using:
- Printed photographs
- Mobile screen replays
- Static images
Current checks:
- Blink
- Smile
- Head Left
- Head Center
- Head Right
Why needed:
A face photo alone should never be enough for authentication.
Purpose:
Capture a high-quality face image after liveness passes.
Main Responsibilities:
- Take photo
- Extract pixel buffer
- Generate face embedding
- Save employee record
- Verify employee identity
Why needed:
This is the entry point into the recognition pipeline.
Purpose:
Crop only the face area from the captured image.
Example:
Original Image
1740 × 2320
Face Region
529 × 528
Why needed:
Removing background improves recognition consistency.
Purpose:
Resize cropped face to:
112 × 112
Why needed:
MobileFaceNet expects a fixed input size.
Without resizing:
Model Inference Fails
Purpose:
Convert face image into a numerical representation.
Input:
112 × 112 RGB Face
Output:
192-Dimensional Embedding
Example:
[
-0.008,
0.038,
0.016,
...
]
Why needed:
Comparing raw images is unreliable.
Embeddings provide a compact mathematical representation of a face.
Purpose:
Generate unique facial features.
Function:
generateEmbedding()
Output:
192 Numbers
Why needed:
These embeddings are stored instead of face images.
Benefits:
- Smaller storage
- Better privacy
- Faster comparison
Purpose:
Enroll new personnel.
Steps:
- Pass liveness
- Capture image
- Generate embedding
- Save locally
Stored Fields:
Employee ID
Employee Name
Embedding
Timestamp
Network Status
Sync Status
Purpose:
Authenticate personnel.
Steps:
- Pass liveness
- Capture face
- Generate embedding
- Load stored embedding
- Compare embeddings
Output:
Score = 0.92
Verified ✓
Why needed:
Authentication is based on mathematical similarity rather than image matching.
Purpose:
Compare two embeddings.
Function:
compareEmbeddings()
Example:
0.95 = Same Person
0.45 = Different Person
Current Threshold:
0.80
Why needed:
Determines whether the captured face belongs to the registered employee.
Purpose:
Store records when internet is unavailable.
Technology:
- AsyncStorage
Stores:
- Employees
- Verification records
Why needed:
The system must work completely offline.
Purpose:
Detect online/offline state.
Technology:
- NetInfo
Outputs:
ONLINE
OFFLINE
Why needed:
Controls sync behavior.
Purpose:
Synchronize records when connectivity returns.
Current Prototype:
Local Records
│
▼
Simulated AWS Upload
│
▼
Local Purge
Why needed:
Required by the hackathon specification.
Future Version:
AWS API
│
▼
Successful Upload
│
▼
Purge Local Records
Advantages:
- Smaller storage
- Faster comparison
- Better privacy
Advantages:
- Reduces spoof attacks
- Avoids unnecessary processing
Advantages:
- Lightweight
- Mobile-friendly
- Fast inference
- Suitable for offline devices
- AWS Integration
- AES-256 Storage Encryption
- Background Sync
- Advanced Anti-Spoofing
- Multi-Face Support
- Benchmarking Dashboard
ML Kit provided face coordinates from the camera preview, but the captured photo had a different resolution.
Example:
Preview:
1080 × 1920
Captured Photo:
1740 × 2320
As a result, the wrong area was cropped and the recognition model received incorrect facial regions.
A coordinate scaling layer was implemented to map face coordinates from preview space into captured image space.
Additional margins were added around the detected face to ensure the complete facial region was included.
- Stable face crops
- Improved embedding consistency
- Better verification accuracy
Initially the entire camera image was passed into the recognition model.
The model received:
Face
Background
Walls
Lighting Variations
which reduced embedding quality.
A face-only crop pipeline was introduced:
Face Detection
↓
Face Crop
↓
112 × 112 Resize
↓
MobileFaceNet
- Cleaner embeddings
- Faster inference
- More reliable verification scores
A printed photograph or image displayed on another device could potentially be presented to the camera.
A multi-step liveness workflow was implemented:
- Blink Detection
- Smile Detection
- Head Turn Left
- Head Center
- Head Turn Right
Authentication requires active user participation rather than a static image.
Remote locations often have no mobile data or unstable network access.
Cloud-based face verification was therefore unsuitable.
All processing was moved on-device:
Face Detection
Liveness Detection
Embedding Generation
Verification
Storage
The complete authentication pipeline functions offline.
Storing face images increases storage usage and privacy risks.
Only 192-dimensional face embeddings are stored.
Face photographs are discarded after processing.
- Reduced storage requirements
- Better privacy
- Faster comparisons
The same employee could accidentally be enrolled multiple times.
Employee IDs are validated before storage.
Duplicate IDs trigger an error:
Employee ID already exists
Consistent employee records.
Authentication records generated offline must eventually reach the backend.
A Sync & Purge workflow was designed:
Local Storage
↓
Cloud Sync
↓
Purge Local Records
Current prototype simulates AWS synchronization.
Architecture is ready for backend integration.
The solution must operate on standard mid-range devices.
MobileFaceNet was selected because:
- Lightweight architecture
- Fast inference
- Mobile optimization
- TensorFlow Lite support
Face embedding generation typically completes in less than one second.
- Single codebase
- Android support
- iOS support
- Faster development
- Offline operation
- Mobile optimized
- Accurate face landmarks
- Lightweight
- Runs on-device
- No server dependency
- Fast inference
- Cross-platform support
- Small model size
- Mobile-friendly
- Suitable for edge AI deployments
- Simple local persistence
- Offline capability
- Lightweight integration
- Detects connectivity restoration
- Enables Sync & Purge workflow
During development, the most difficult part was not face recognition itself, but ensuring that:
- Face crops remained consistent across different camera resolutions.
- Liveness verification prevented spoofing attempts.
- The complete pipeline remained functional without internet connectivity.
- Processing remained lightweight enough for mid-range mobile devices.
The project demonstrated that reliable offline biometric authentication can be achieved entirely on-device using open-source technologies.