ON INFORMATICS VISUALIZATION

— Recently, researchers focused on face image manipulation detection and localization techniques because of their importance in image security applications. The previous research has not highlighted the recovery of the face region after manipulation detection. This paper presents a new face region recovery algorithm (FRRA) to be included in the face image manipulation detection algorithms (FIMD). The proposed FRRA consists of two main algorithms: face data generation algorithm and face region restoration algorithm. Both algorithms start by detecting the face region using Multi-task Cascaded Neural Network followed by a face window selection process. In the face data generation algorithm, the recovery information is generated from the shirked face window using bicubic interpolation technique. In the face region restoration algorithm, the face region zoomed using bicubic interpolation technique. The proposed FRRA has been tested and compared with previous recovery methods for different color face images, and the results proved that the FRRA could recover the face region with better visual quality at the same data length compared to previous methods. The main contributions of this research are a) the suggestion of including a face region recovery algorithm to FIMD, b) the study of previous recovery data generation algorithms for color face images, and c) introducing a new algorithm for generating the recovery data based on bicubic interpolation. In the future, the proposed algorithm can be included in the recent FIMD algorithms to recover the face region, which can be very useful in practical applications, especially those used in data forensics systems.


I. INTRODUCTION
Authors Digital face images are increasingly used and shared via networks for different purposes.The shared face images can increase the speed of various processes and documentation, facilitate working with different systems, overcome distance restrictions, share memories and personal information, access online applications, and many others.The shared face images can easily be manipulated using digital image processing applications, and the manipulations can be harmful or harmless based on the intentions of the manipulator [1].The recent interest of the research community has been directed towards a type of manipulation called DeepFakes which utilizes deep-learning or machinelearning-based algorithms to create fake face images or videos that are hard to be recognized as fake information [2]- [6].The images and videos generated from DeepFakes can be used for many harmful intentions which may threaten the lives of people and their financial or social situation [7]- [12].
Recently, several researchers introduced face image manipulation detection (FIMD) algorithms to check the authenticity of the image [13]- [18].Most available techniques have been implemented based on machine learning or deep learning algorithms because they depend on the type of manipulation utilized to generate the fake image.These techniques can successfully detect specific types of manipulation under specific conditions, but they suffer from some limitations and may face several limitations.Some recently published review papers have been presented to illustrate the types of FIMD, their limitations, and challenges [19]- [23].The review paper in [23] highlighted some of the challenges that can face FIMD techniques, such as: (a) the rapid development of face image processing applications, (b) the requirement of large and high-quality datasets for training, (c) the need for knowing the type of manipulation applied in order to choose the suitable detection technique, (d) the lack of generalization, (e) the lack of standard metrics, (f) the high complexity and time-consuming process required for training networks, (g) the generation of high-quality fake face images which are difficult to be detected by the trained network, in addition to other challenges.Some solutions have been suggested to overcome the limitations and challenges [23].One suggested strategy is to implement a face image authentication technique based on image watermarking.
A new FIMD technique based on face detection and image watermarking algorithms [24].The face region is detected, and a binary mask image is generated to classify the image blocks into two groups belonging to the face region or outside the face region.The manipulation reveals data are extracted from the blocks that belong to the face region and embedded in the blocks outside the face region using Slantlet-based image watermarking [25]- [27].The proposed technique outperforms previous detection techniques, which can detect and localize different face image manipulations with 100 % accuracy [24].However, the face region's recovery has not been highlighted in this technique.It will be very useful for security and forensics applications to recover the original face region if manipulations exist.We suggest adding a data generation and recovery algorithm to the steps of the FIMD technique [24].
The recovery information has been generated using the average of 4×4 blocks of pixels [28]- [30].At the receiver side, the average value replaces the pixels in its related 4×4 block.The recovery information has been generated using the average of 2×2 blocks of pixels [31].At the receiver side, the average value replaces the pixels in its related 2×2 block.The method by Zain and Fauzi [31] recovered the image with better visual quality results compared to the average 4×4 method.However, it was at the cost of increasing the number of bits generated in the recovery information.An Integer Wavelet Transform (IWT) based recovery information generation method has been applied [32].Some other studies have been used to generate recovery information for grayscale medical images [28]- [32].
In this paper, a new face region recovery algorithm (FRRA) is presented in order to be included in the face image manipulation detection algorithms (FIMD).The proposed FRRA consists of two main algorithms: face data generation algorithm and face region restoration algorithm.Both algorithms start by detecting the face region using Multi-task Cascaded Neural Network (MTCNN) followed by the face window selection process.In the face data generation algorithm, the recovery information is generated from the shirked face window using the bicubic interpolation technique [33], [34].In the face region restoration algorithm, the face region is zoomed using the bicubic interpolation technique.In order to compare the performance of the proposed FRRA with the previous methods, the average 2×2 and IWT-based algorithms have been applied to color face images.The main contributions of this work can be summarized as follows:  The suggestion of including a face region recovery algorithm in the available FIMD techniques is a new idea that has not been highlighted in previous FIMD methods. The study of two previous recovery data generation algorithms for color face images. Introducing a new algorithm for generating the recovery data from the face region based on bicubic interpolation.
The rest of the paper is organized as follows: section 2 presents the related works; section 3 presents the proposed FRRA; section 4 illustrates the experimental results and discussion; finally, section 5 presents the conclusions of this work.

II. MATERIALS AND METHOD
As mentioned in the introduction, the average 2×2 and IWT algorithms have been applied to generate the recovery information of the region of interest in grayscale medical images.The proposed research in this paper aims to introduce a face region recovery algorithm for color images.Therefore, the average 2×2 and IWT-based algorithms have been implemented to serve the aim of this research.The proposed FRRA consists of two main algorithms: the face data generation algorithm, which is applied on the sender side, and the face region restoration algorithm, which is applied on the receiver side.
The following subsections illustrate the steps of the implemented algorithms based on average 2×2 and IWT.To test the visual quality of the recovered face region, the Peak Signal-to-Noise Ratio (PSNR) and the Structured Similarity Index (SSIM) between the original face region and the recovered face region have been calculated.Then the block diagrams of the proposed FRRA algorithms and their details are explained.

A. Generating Recovery Data based on Average 2×2 Block
The proposed algorithms of generating the recovery information based on dividing the face region into nonoverlapping blocks, each of size 2×2 pixels are illustrated in 2) The steps of average 2×2-based recovery algorithm at the receiver side are as follows:   2) The steps of the IWT-based recovery algorithm at the receiver side:

C. The proposed FRRA algorithms
The proposed algorithms for generating the recovery information based on the bicubic interpolation technique are explained in the following subsections.2) Face region restoration algorithm: The input of the proposed algorithm for face region restoration is the binary sequence, while the output of this algorithm is the recovered face region.The block diagram of this algorithm is shown in Figure 4, and the steps of the algorithm are as follows:

A. Test of Average 2×2 algorithm
In this experiment, the visual quality of the recovered face region using the average 2×2 algorithm is calculated using PSNR and SSIM.Table II presents

B. Test of IWT Algorithm
In this experiment, the visual quality of the recovered face region using IWT algorithm is calculated using PSNR and SSIM.Different IWT types have been tested to choose the one with the best results.Table III presents samples of the PSNR results using different IWT types.The results proved that the best type of IWT is cdf 3.5.Table IV

D. Comparison between the proposed algorithms
This section presents a comparison of visual quality for average 2×2 (AVG 2×2), IWT (cdf 3.5), and FRRA.

IV. CONCLUSION
The recent interest of the research community has been directed toward face image manipulation detection (FIMD) techniques where different algorithms have been presented.Recently, the watermarking based FIMD proved its efficiency in detecting and localizing various manipulations in the face image.However, the recovery of the original face image has not been highlighted.To improve the performance of the watermarking based FIMD technique, this paper highlighted the need for recovering the face region if manipulations exist using recovery algorithms based on average (2×2), IWT (cdf 3.5), and bicubic interpolation.The visual quality of the recovered face images using the proposed algorithms has been tested using PSNR and SSIM.The results proved that the proposed face region recovery algorithm using bicubic interpolation obtained the best results and can be considered in the future to be included in the FIMD technique.
Fig 1 and can be summarized as follows: 1) The steps of the average 2×2-based recovery algorithm at the sender side:  Step 1: Read the original face image. Step 2: Apply MTCNN to detect the face box. Step 3: Adjust the output of the MTCNN to select the face region. Step 4: Divide one channel from the face region into non-overlapping blocks of size 2×2 pixels. Step 5: Calculate the average of each block from Step 4.  Step 6: Convert the average values to binary and concatenate the bits to generate one binary sequence. Step 7: Repeat step 4 to step 6 for the remaining channels of the face region. Step 8: Calculate the length of the resultant binary sequences.

 Step 1 :
Read the binary sequence for one channel. Step 2: Divide the binary sequence into nonoverlapping subsequences, each of length (8 bits). Step 3: Convert the binary subsequences to decimal to recover the average values. Step 4: Recover the channel of the face region by replacing the pixels in each (2×2) block with their average value. Step 5: Repeat step 1 to step 4 to recover the other two channels of the face region. Step 6: Calculate the PSNR and SSIM to test the visual quality of the recovered face region.

Fig. 1
Fig. 1 Recovery algorithm based on average (2×2) block.B.Generating Recovery Data based on IWTThe proposed algorithms for generating the recovery information based on IWT are illustrated in Fig2and can be summarized as follows:

 Step 1 :
Read the binary sequence for one channel. Step 2: Divide the binary sequence into subsequences, each of length (8 bits). Step 3: Convert the binary subsequences to decimal to recover CA coefficients. Step 4: Set CH, CV, and CD to zeros. Step 5: Apply inverse IWT to recover the channel image. Step 6: Repeat steps 1 to 5 to the remaining two channels. Step 7: Calculate the PSNR and SSIM to test the visual quality of the recovered face region.

1 )
Face data generation algorithm: The input of the proposed algorithm for face data generation is the original face image, while the output of this algorithm is the generated binary sequence.The block diagram of this algorithm is shown in Fig 3, and the steps of the algorithm are as follows:  Step 1: Read the original face image. Step 2: Apply MTCNN to detect the face box. Step 3: Adjust the output of the MTCNN to select the face region. Step 4: Apply bicubic interpolation with a scale value of 0.5. Step 5: Select one channel from the face region and convert it to binary, then concatenate the binary bits to obtain one binary sequence. Step 6: Repeat step 5 to the remaining two channels and calculate the length of the resultant binary sequences.

 Step 1 :
Read the binary sequence for one channel. Step 2: Divide the binary sequence into subsequences, each of length (8 bits). Step 3: Convert the binary subsequences to decimal to recover resized channel. Step 4: Repeat steps 1 to 3 to recover the remaining two channels. Step 5: Apply bicubic interpolation with scale value (2) to the recovered resized face region. Step 6: Calculate the PSNR and SSIM to test the visual quality of the recovered face region.

Fig. 4
Fig. 4 Proposed face region restoration algorithm.III.RESULTS AND DISCUSSION Different test images have been used with different sizes to test the proposed algorithms' performance.Samples of the test images are shown in Fig 5, and their corresponding sizes are shown in Table I.The experiments have been conducted to test the performance of average 2×2, IWT, and the proposed FRRA.PSNR and SSIM have been calculated to test the recovered face region's visual quality.The length of the binary sequence (Lseq) is also calculated to illustrate the number of recovery bits that are generated from each algorithm.The following subsections present the experiments and their results.
the results for the ten face images shown in Fig 5. Samples of the recovered face regions using this algorithm are shown in Fig 6.
presents the experimental results for IWT cdf 3.5 for the test images shown in Fig 5. Samples of the recovered face regions using this algorithm are shown in Fig 7.

TABLE II TEST
OF VISUAL QUALITY AND LENGTH OF SEQUENCE FOR AVERAGE (2×2)

TABLE III TEST
OF VISUAL QUALITY FOR DIFFERENT IWT TYPES

TABLE IV TEST
OF VISUAL QUALITY AND LENGTH OF SEQUENCE FOR IWT(CDF 3.5)In this experiment, the visual quality of the recovered face region using FRRA is calculated using PSNR and SSIM.TableVpresents the results for the ten face images shown in Fig 5. Samples of the recovered face regions using this algorithm are shown in Fig 8.

TABLE V TEST
OF VISUAL QUALITY AND LENGTH OF SEQUENCE FOR FRRA Table VI and Table VII compare PSNR results for the ten test images shown in Fig 5, which proved that the best visual quality of the recovered face region has been obtained using FRRA.

TABLE VII COMPARISON
OF SSIM