- 1 -
中国科技论文在线
一种安全的人脸识别方法#
王晨光，闫达远*
Foundations: This work is supported by a grant from the National High Technology Research and Development
Program of China (863 Program) (No. 2009AA01Z433)
（软件学院，北京理工大学，北京 100081）
摘要：研究结合私密人脸识别，辅助解决云计算安全性问题的方法。方法分为三部分：用户
端部分提供人脸图片；云端初始化部分提供人脸子空间和用于匹配的人脸模板库；云端私密
识别匹配部分提供方法的核心算法，即安全的比较已被加密的数大小。实验结果说明，方法
能够保证云端即不知道用户的真实人脸数据，也不知道私密识别匹配结果，云端与用户端的
交互过程安全，使得用户端人脸数据得到安全性保障，提供了一种可信、高效、复杂性低的
安全云计算方法。
关键词： 人工智能；私密识别匹配；人脸识别
中图分类号：TP393
A Secure Approach for Face Recognition
WANG Chenguang, Yan Dayuan
(School of Software,Beijing Institute of Technology, Beijing 100081)
Abstract: Supporting study of a method to solve cloud computing security issue with private face
recognition. The method has three parts: user part provides face images; cloud initialization part has a
face subspace and templates database; cloud private matching identification part contains the core
algorithm of the method, comparing two encrypted numbers under double-encrypted conditions. The
experimental results show the method can ensure that cloud neither know user’s real face data, nor the
face private matching identification result, to make user’s face data secure, we develop a credible,
efficient, low-complex method to guarantee cloud computing security.
Key words: artificial intelligence; private matching identification; face recognition
0 Introduction
Cloud computing is the network trend, with which people's lives have more relationship.
However, a major characteristic of cloud computing is distributed computation based on unfixed
nodes, operations often carried out without trusted nodes, so the calculation involved with user
privacy information is insecure.
In this paper, we focus on how to solve the security issues of cloud computing. Cloud
computing security based on private face recognition ‘s significance is that face recognition will
be applied to the cloud computing for the first time, supporting proof of private matching
identification resolves security issues of cloud computing credibly, efficiently. Calculation of face
recognition and matching is under encrypted conditions, user sends a double encrypted face image
to cloud, and cloud operates face recognition and matching under the encrypted conditions, the
result is encrypted again before encrypted transmission to user [1]. In this way, cloud neither knows
user’s real face data, nor which face and the face matches in templates, ensure no leakage of user
privacy data.
The remainder of the paper is organized as follows. In Section 2 we provide a brief overview
of the related work. In Section 3, we give detailed analysis of cloud computing security based on
private face recognition method. Section 4 shows our experimental data and results .Our
- 2 -
中国科技论文在线
conclusion is provided in Section 5.
1 Related Work
The solution to the issue needs three-part collaboration; therefore, secure multi-collaborative
computing is needed. Yao [2] introduced the basic concepts of secure computing. Later, people
provided a lot of secure calculation solutions, namely combinatorial circuits [3, 4], ordered binary
decision diagrams [5], branching programs [6, 7], or one-dimensional look-up table [6].However, the
computational complexity of these methods are too high to meet the paper, for the considering of
combining biometric with cloud computing. Therefore, specific methods must be improved.
Some people try to use the private biometric matching identification, especially in fingerprint
and iris [8, 9, 10].However, these show more concerns on hardware architecture, such as biological
data hash template is stored on the server. Server can know the result of matching (to only ensure
the template is stored securely). In contrast, our scenario allows hide this information, and apply
it to cloud computing .As far as we know, there is no helpful solution to solve the problem, when
cloud computing involved with biometrics, efficiency and security problems appears.
2 Approach
Assume that cloud is B, user is A. The diagram of our approaching method is summarized in
Fig. 1.
图 1 方法概要图
Fig. 1 Method diagram
Our method is divided into three parts: user, cloud initialization and private matching
identification part of cloud. User part uses a series of face preprocessing method to do with
original images, using Paillier [11] encryption algorithm encrypt processed images; cloud
initialization part uses the processed original images to establish subspaces and face templates
database through PCA [12] algorithm; cloud private matching identification part has projection,
distance calculation, minimum distance finding [13] combined to achieve a face matching and
recognition under encrypted conditions; cloud and user’s communication is also in encrypted
domain. Experimental results show that the method is credible and efficient to support cloud
computing security study.
User Part
A reads the original image, firstly preprocessing , then face detection and graying, also face
vectoring ,after double-encrypt each pixel data ,data sent to B. Processing diagram shown in Fig. 2
below:
- 3 -
中国科技论文在线
图 2 用户端处理流程
Fig. 2 User processing diagram
Preprocessing including image light, color, size, etc. makes the input of each original image
uniform and consistent; face detection and graying contain finding the face region from original
image, and cutting face down in unifying size, then convert each pixel’s RGB three-color value to
gray scale data; face vectoring transforms the two-dimensional face image to one-dimensional
vector, Denoted as Γ , Double encryption firstly use Paillier encryption algorithm to encrypt each
pixel’s gray value, following paper presents "[]" on behalf of Paillier encryption process,
Encryption will be denoted by vector[ ]Γ .Then, use Elliptical encryption, Denoted by [[ ]]Γ ,
represent Elliptic encryption process in following paper with "[[]]".Finally, [[ ]]Γ is passed to B.
Encryption algorithm uses Paillier, because the Paillier encryption algorithm is additively
homomorphic, and the encryption process is more simple and efficient. Paillier encryption
algorithm is additively homomorphic because: [ ] [ ] [ ]a b a b⋅ = + , further: [ ] [ ]bab a= . Cloud
private matching identification are based on the above two properties.
Using Elliptic encryption for the distributed computation and poor security when
communicating with cloud computing. Because the group protocol based on Elliptic encryption
enables cloud and user’s communication data secure, credible, and complete when in an insecure,
open network communication environment. Elliptic encryption is described as follows:
selects an Elliptic curve Ep(a,b), y2=x3+ax+b(mod p), and get a point on the Elliptic
curve as point G.
selects a private key k, and generates public key K=kG.
Step3. A sends Ep(a,b)and point K，G to B.
B received the information, it will be encoded to be transmitted to the point M
on Ep(a,b), and generates a random integer r(r<n).
calculates points C1=M+rK; C2=r.
passes C1、C2 to A.
receiving the information, A calculates C1-kC2; the result is the point M.
Because C1-kC2=M+rK-k(rG)=M+rK-r(kG)=M, then the point M can be explicitly decode.
Cloud Initialization Part
The role of the part is to establish face subspace and the matching face templates database.
Suppose there are M face images for matching. After a series of preprocessing described
above, like detection, graying, and vectoring to get M face vectors, denoted as 1 2, , ..., Mθ θ θ . Using
PCA algorithm, the input data X is M individual face vectors, then obtain eigenvector matrixW ,
set 1 2[ , ,..., ]MW µ µ µ= , the matrixW ’s column k denoted as kµ .
Use the formula ( )Ty W x µ= − to get projection coefficient of each face templates image iθ ,
denoted as
iΩ and 1 2[ , ,..., ]Ti i i iMω ω ωΩ = .
- 4 -
中国科技论文在线
Pass the feature vector matrix W and the projection of each face templates’ coefficient iΩ to
private matching identification part of cloud.
In order to ensure private matching identification simple, the mean face needs calculated,
denoted byψ , is defined as
1
1 M
iiM
ψ θ== ∑ . Finally, pass the mean face to private matching
identification part.
Cloud Private Matching Identification Part
This part is the core of B, achieving face matching recognition in encrypted domain, using
Paillier encryption algorithm and Elliptic encryption algorithm for double encryption.
This section is divided into three steps, namely, projection, distance calculation, minimum
distance finding.
Projection
This step is to project the high-dimensional original data into lower-dimensional subspace,
then obtain the projection coefficients of original face. Set data received by B is[ ]Γ .
In the case of non-encrypted condition, firstly, using original one-dimensional face vector
subtract the average face, namely:
1
.
.
.
N N
ψ
ψ
ψ
Γ −⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟Φ = Γ − = ⎜ ⎟⎜ ⎟⎜ ⎟Γ −⎝ ⎠
(1)
Then project to the subspace, namely:
TWΩ = Φ (2)
Where 1 2[ , ,..., ]Mω ω ωΩ = and 1 1 2 2Ti i i i N iNω µ µ µ µ= ⋅Φ = Φ ⋅ + Φ ⋅ + ⋅⋅⋅+ Φ ⋅ is the projection
coefficient of input face and also a computing base for the following distance calculation.
But for B, to protect user privacy, the operation must be carried out in the encrypted domain.
Because Paillier encryption algorithm is additively homomorphic, the following operations
happen:
1 1[ ] [ ]
.
[ ] [ ] .
.
[ ] [ ]N N
ψ
ψ
ψ
Γ ⋅ −⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟Φ = Γ − = ⎜ ⎟⎜ ⎟⎜ ⎟Γ ⋅ −⎝ ⎠
(3)
So B's projection calculation becomes:
1
1 1
1
[ ] [ ]
[ ] ... [ ] , 1, 2,...,i iN
i i N iN
N i M
µ µ
ω µ µ= Φ ⋅ + ⋅⋅⋅ + Φ ⋅
= Φ ⋅ ⋅ Φ =
(4)
After the M times’ operation, B can receive encrypted projection coefficient[ ]Ω .
As B knows Ψ and each iµ , the operation is very convenient. More importantly, these
operations are without A, face templates database will not be leaked to A. Operation of both sides
doesn’t need the other’s participation, privacy-information security will be guaranteed.
Distance Calculation
After receiving the input encrypted face projection coefficient [ ]Ω , calculate the distance
- 5 -
中国科技论文在线
between the input face and each template in face templates database. Distance defined as:
2 2
11
22
1 1 1
1 2 3
( , ) || ||
( ) ( )
( 2 )
kk
K K K
i ii i
i i i
D
S S S
ω ω ω ω
ω ω ω ω
= = =
Ω Ω = Ω − Ω
= − + ⋅⋅⋅+ −
= + − +
= + +
∑ ∑ ∑
(5)
Distance formula is transformed into three-part, 1S , 2S and 3S . Where 21
1
K
i
i
S ω
=
= ∑ ,
2
1
( 2 )
K
ii
i
S ω ω
=
= −∑ , 23
1
K
i
i
S ω
=
= ∑ .In the encrypted domain, distance calculation becomes:
1 2 3[ ( , )] [ ] [ ] [ ]D S S SΩ Ω = ⋅ ⋅ (6)
It's easy to B to compute 1S , because the projection coefficient iω of face templates is
already known. B needs to calculate 1S first, then encrypted by A’s public key. 2S ’s calculation
follows the formula:
2
2
1 1
[ ] [ ( 2 )] [ ]
iKK
i ii
i i
S
ω
ω ω ω
−
= =
= − =∑ ∏ (7)
For 3S , the computation is slightly complex, which requires B and A’s collaboration. First, B
generates a random number iϒ for each iω followed with Paillier encrypted, then
calculates [ ] [ ] [ ] [ ]i i i i ix ω ω= + ϒ = ⋅ ϒ , transform into [[ ]]ix using Elliptic encryption. Random
number
iϒ can increase ambiguity, so transmission doesn’t leak. Later, B sends M [[ ]]ix to A.
[[ ]]ix decrypted by A with their own private key to obtain ix and
2
ix , double encrypted to 2[[ ]]ix ,
then pass to B. Following diagram Fig. 3 show the transmission:
图 3 双重加密示意图
Fig. 3 Double encrypted transmission diagram
When B obtain
2[[ ]]ix , decrypt to
2[ ]ix , then process obey below:
2
22 2 2 2[ ] [ ] [ ] [( ) 2 ]
[ ]
ii i ii i i i i
i
x ω ω ω
ω
− ϒ⋅ ⋅ −ϒ = + ϒ − ϒ − ϒ
= (8)
Later, multiply together each 2[ ]iω to compute 3[ ]S . Note’s calculation performs only once.
The computation of distance between each template and input face can directly use 3[ ]S .
Each face template performs the above algorithm to obtain each template’s distance with the
input face[ ] [ ( , )]i iD D= Ω Ω . The implementation of the algorithm is in the encrypted domain.
Minimum Distance Finding
When the distances [ ]( 1,2,..., )iD i M= are calculated complete, begin to find the shortest
- 6 -
中国科技论文在线
distance among M encrypted distances. Tree structure is used to obtain the minimum distance , M
distances are first divided by even and odd neighboring into M/2 groups, each group will leave the
smaller one, reject the bigger one, then remain M/2 distances. Follow the above flow, the
minimum distance can be found.
The key to the issue is to compare[ ]iD and. In other words, to compare the two encrypted
numbers [ ]a and[ ]b . To solve the issue, the algorithm is as follows:
produces a random number r, encrypted to[r];
passes [ ] [ ][ ]a a+ ϒ = ϒ and [ ] [ ][ ]b b+ ϒ = ϒ to A;
decrypt, obtain a+r and b+r, subtract the two numbers, if result is negative,
then 1ϒ = ,otherwise 0ϒ = ;
Step4. A passes [ ]ϒ to B;
Step5. B brings [ ]ϒ to the following formula:
[ ]/[ ][ ] [ ] [( ) ( ) ] [ ]a b b a b a b b mϒ ⋅ = < ⋅ − + = (9)
The result [m] is the smaller one of a and b‘s cipher text, show the credible, efficient result of
comparing two encrypted numbers.
Finally, Elliptical encryption algorithm encrypts the smaller number based on the above
method, obtain [[m]], then pass [[m]] back to A, A uses private key to decrypt [[m]], soon knows
result.
3 EXPERIMENTS
Dataset
We use "ORL Database of Faces" [14], a widely used image database in the experiment, The
following Tab. 1 show how we use the data:
表 1 数据分布
Tab. 1 Dataset Distribution
Image Type Quantity Image In Individual In
Cloud Database 10*5 Yes Yes
User images 1 10*3 No No
User images 2 10*3 Yes Yes
User images 3 10*3 No Yes
“Cloud Database” is the face templates database of Cloud, the rest image sets is used to test
our method.“Image In” infers “Image in Templates Database”, and” Individual In” is short for
“Individual in Templates Database”.
Results
We use the above dataset; the final results are as follows Tab. 2:
表 2 实验结果
Tab. 2 Experimental Results
Image Type Encryption /
Initialized Data
Total Time
Consuming
Recognition Rate
Cloud Database -10051685 -485… <>15s -
User images 1 19258160296… <>21s %
User images 2 12065752040… <>16s %
User images 3 14172822945… <>17s %
- 7 -
中国科技论文在线
Combined with the above results, analysis the algorithm from two sides as follows:
Credibility
By credibility testing, we find the recognition rate satisfied. The traditional PCA face
recognition algorithm’s recognition rate is 95%, so with the combination of double encryption
algorithm and cloud computing model, face recognition accuracy does not decrease obviously,
which makes the credibility of our algorithm guaranteed. However, after deep analysis, along with
the image stretching factor changing, varieties in recognition rate are shown in Tab. 3:
表 3 伸缩因子与识别率关系
Tab. 3 Relation between Stretching and Recognition Rate
Stretching Factor Recognition Rate
0 %
1 %
2 %
3 %
4 %
5 %
With the base-10 stretching factor, when the factor increases to 3, the recognition rate
remains at 93%, Tab. 2 default set factor to 3, not only can this factor be able to ensure the
recognition rate but also reduce the complexity of the algorithm.
Complexity of the algorithm
We can analysis the algorithm time complexity through Tab. 2 directly, with the change of
image type, algorithm time consuming varies, in different cases, the private matching
identification has different time-consuming when finding the minimum distance. Following Tab. 4
shows:
表 4 私密识别时间
Tab. 4 Private Time Consuming
Image Type Private Time Consuming
User images 1 <>16s
User images 2 <>12s
User images 3 <>13s
Complexity of the algorithm varies when the dimension generated by the images’
lower-dimensional process differs. As Tab. 2 shows above, it is much more time-consuming
because both images’ encryption and initialization require large number operation. Tab. 5 show:
表 5 与维数的关系
Tab. 5 Relation With Dimension
Dimension Total Time Consuming
92*112 <>240s
10*10 <>60s
1*10 <>15s
With the algorithm’s credibility and complexity analysis, the experimental results can ensure
the credibility, efficiency, low-complexity of the private matching identification, and support the
study of cloud computing security.
4 Conclusion
This paper focuses on the fast-developing cloud computing security issue, combined with
face recognition, presents a creative method called cloud computing security based on private face
- 8 -
中国科技论文在线
recognition, which is a way to solve the issue. The core of the method is proposed to compare
numbers in the encrypted domain, allow user obtain same, correct result as under non-encrypted
conditions. The method proves to be credible, efficient, low-complex, and supports further study
of cloud computing security.
As the future work, we now use PCA algorithm for face recognition, and algorithm having
higher recognition rate appears, due to the higher complexity of these algorithms, it’s difficult to
apply to encrypted domain, so we leave this as our future work; and we will modify the
implemented algorithms using multiple threads to improve performance of the algorithms.
References
[1] Blake, ., Kolesnikov, V.: Conditional Encrypted Mapping and Comparing Encrypted Numbers. In: Di
Crescenzo, G., Rubin, A. (eds.) FC 2006. LNCS, vol. 4107, pp. 206–220. Springer, Heidelberg (2006).
[2] Yao, .-C.: Protocols for Secure Computations (Extended Abstract). In: Annual Symposium on
Foundations of Computer Science – FOCS 1982, November 3-5,pp. 160–164. IEEE Computer Society Press,
Los Alamitos (1982).
[3] Goldreich, O., Micali, S., Wigderson, A.: How to Play any Mental Game or A Completeness Theorem for
Protocols with Honest Majority. In: ACM Symposium on Theory of Computing – STOC 1987, May 25-27, pp.
218–229. ACM Press, New York (1987).
[4] Jacobsson, M., Juels, A.: Mix and match: Secure function evaluation via : Okamoto, T. (ed.)
ASIACRYPT 2000. LNCS, vol. 1976, pp. 162–, Heidelberg (2000).
[5] Kruger, L., Jha, S., Goh, E.-J., Boneh, D.: Secure function evaluation with ordered binary decision diagrams.
In: Proceedings of the 13th ACM conference on Computer and communications security CCS 2006, Virginia,
, pp. 410–420. ACM Press, New York (2006).
[6] Naor, M., Nissim, K.: Communication complexity and secure function evaluation. Electronic Colloquium on
Computational Complexity (ECCC), 8(062) (2001).
[7] Naor, M., Nissim, K.: Communication preserving protocols for secure function evaluation. In: ACM
Symposium on Theory of Computing, pp. 590–599 (2001).
[8] Kevenaar, T.: Protection of Biometric Information. In: Security with Noisy Data,pp. 169–193. Springer,
Heidelberg (2007).
[9] Ratha, N., Connell, J., Bolle, R., Chikkerur, S.: Cancelable biometrics: A case study in fingerprints. In:
Proceedings of the 18th International Conference on Pattern Recognition (ICPR), vol. IV, pp. 370–373. IEEE
Press, Los Alamitos (2006).
[10] Tuyls, P., Akkermans, ., Kevenaar, ., Schrijen, G.-J., Bazen, ., Veldhuis, .: Practical
biometric authentication with template : Kanade, T., Jain, A., Ratha, . (eds.) AVBPA 2005.
LNCS, vol. 3546,pp. 436–446. Springer, Heidelberg (2005).
[11] Damg˚ard, I., Jurik, M.: A Generalization, a Simplification and some Applications of Paillier’s Probabilistic
Public-Key System. Technical report, Department of Computer Science, University of Aarhus (2000).
[12] Turk, ., Pentland, .: Face recognition using eigenfaces. In: IEEE Computer Society Conference on
Computer Vision and Pattern Recognition, pp. 586–591(1991).
[13] Z. Erkin, M. Franz, J. Guajardo, S. Katzenbeisser, I. Lagendijk, and T. Toft, “Privacy-preserving face
recognition,” Privacy Preserving Technologies, LNCS, vol. 5672, pp. 235–253, 2009.
[14] The Database of Faces, (formerly‘The ORL Database of Faces’) AT&T Laboratories Cambridge:
