Sensitive Field Mask Processing in System Logs

For the development of financial business, many interfaces need to use user information, and there will inevitably be some sensitive fields in user information, such as user name, bank card number and so on.Therefore, when users save sensitive information and log printing, they cannot save sensitive information in clear text.When a database stores user-sensitive information, there is usually an encryption/decryption system in the system.User information is encrypted and saved to the database when sensitive information needs to be saved.This is not a topic discussed in this article, and the following formats are recommended when saving in a database:

User ID number: 511911202005281234
 Encrypted data: P1234567
 Data saved in DB: P1234567:511******1234

It is possible to use masked information when comparing user information to see if the real ID number of member information can correspond.

1. Issues to Consider for Log Printing

Let's talk about sensitive log printing in the system.

When we need to print logs, we usually print them in two ways.

  • Rewrite the toString of Object to print information about fields of interest
  • Calling fastjson (or other JSON processing framework)JSON.toString Method

In addition, the following two processing methods are commonly used for log processing:

  • Ignore logs, field logs can be ignored when this field does not affect log queries.Do not print when the object field is empty (or null values), and print *** when there are values.
  • Regular printing of logs can be used when this field affects log queries.For example, the user's ID number can be printed in the log in the first three digits, and the last four digits in the middle plus six *numbers.For example: 511******1234.

Based on the above considerations, we will achieve it.

2. Object toString Printing

2.1 Ignore

Label a comment to indicate that the log for this field in the object can be ignored.

@Target({ElementType.FIELD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface Ignore {
}

2.2 Mask

Label a comment indicating that the log for this field in the object affects the query for the log and can be printed regularly.

@Target({ElementType.FIELD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface Mask {

    String pattern() default "";

}

2.3 AnnotationToStringBuilder

Implement ReeflectionToStringBuilder in common-lang3 to invoke the logs in the printed object through reflection.Overrides the ReflectionToStringBuilder#appendFieldsIn method to achieve our print results.

package com.ushareit.fintech.framework.log;

import com.ushareit.fintech.framework.log.annotation.Ignore;
import com.ushareit.fintech.framework.log.annotation.Mask;
import com.ushareit.fintech.framework.utils.OutMaskUtil;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.builder.ReflectionToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
import org.springframework.core.annotation.AnnotationUtils;

import java.lang.reflect.AccessibleObject;
import java.lang.reflect.Field;

/**
 * <p>Annotated ToString</p>
 *
 * @author zhaoyong
 * @version $Id: AnnotationToStringBuilder , v 0.1  K555 Exp $
 * @date 2018 March 1219:54
 */
public class AnnotationToStringBuilder extends ReflectionToStringBuilder {

    public AnnotationToStringBuilder(Object object) {
        super(object);
    }

    public AnnotationToStringBuilder(Object object, ToStringStyle style) {
        super(object, style);
    }

    public AnnotationToStringBuilder(Object object, ToStringStyle style, StringBuffer buffer) {
        super(object, style, buffer);
    }

    public <T> AnnotationToStringBuilder(T object, ToStringStyle style, StringBuffer buffer, Class<? super T> reflectUpToClass, boolean outputTransients, boolean outputStatics) {
        super(object, style, buffer, reflectUpToClass, outputTransients, outputStatics);
    }

    public <T> AnnotationToStringBuilder(T object, ToStringStyle style, StringBuffer buffer, Class<? super T> reflectUpToClass, boolean outputTransients, boolean outputStatics, boolean excludeNullValues) {
        super(object, style, buffer, reflectUpToClass, outputTransients, outputStatics, excludeNullValues);
    }

    @Override
    protected void appendFieldsIn(Class clazz) {
        if (clazz.isArray()) {
            this.reflectionAppendArray(this.getObject());
            return;
        }
        Field[] fields = clazz.getDeclaredFields();
        AccessibleObject.setAccessible(fields, true);
        for (int i = 0; i < fields.length; i++) {
            Field field = fields[i];
            String fieldName = field.getName();
            if (this.accept(field)) {
                try {
                    Object fieldValue = this.getValue(field);
                    Mask mask = AnnotationUtils.getAnnotation(field, Mask.class);
                    if( (fieldValue instanceof String) && mask != null && StringUtils.isNotBlank(mask.pattern())) {
                        String value = String.class.cast(fieldValue);
                        String pattern = mask.pattern();
                        this.append(fieldName, OutMaskUtil.replaceWithMask(pattern, value));
                        continue;
                    }
                    Ignore ignore = AnnotationUtils.getAnnotation(field, Ignore.class);
                    if(ignore != null) {
                        if(fieldValue != null) {
                            this.append(fieldName, "***");
                        } else {
                            this.append(fieldName, "null");
                        }
                        continue;
                    }
                    this.append(fieldName, fieldValue);
                } catch (IllegalAccessException ex) {
                    throw new InternalError("Unexpected IllegalAccessException: " + ex.getMessage());
                }
            }
        }
    }
}

This requires the implementation of the Object's toString method.For example:

@Data
public class UserInfo {

	/** User Name */
	private String username;

	/** ID number */
	@Mask(pattern = "[\\w]{5}([\\w]*)[\\w]{3}")
	private String idNo;

	/** User Address */
	@Ignore
	private String address;
	
	@Override
	public String toString(){
		return new AnnotationToStringBuilder(this).toString();
	}

}

3. toJSONString method of JSON

In this case, there are two situations, one is that the object is a java bean object, the other is JSONObject.They can all be provided by implementing fastjsonCom.alibaba.fastjson.Serializer.ValueFilterTo process.For Java beans that can be processed through reflection, processing JSONObjects requires special processing and cannot be generalized.

public class LoggerJSON {

	static final SerializeConfig                    SERIALIZE_CONFIG;
	static final MaskFilter                         MASK_FILTER;
	static final Map<String, MaskStrategy>          MASK_FIELDS;

	static {
		SERIALIZE_CONFIG = new SerializeConfig();
		MASK_FILTER = new MaskFilter();
		MASK_FIELDS = buildMaskConfig();
	}

	/**
	 * Mask configuration
	 */
	public static Map<String, MaskStrategy> buildMaskConfig() {
		Map<String, MaskStrategy> result = new HashMap<>();
		result.put("idNo", new NullMaskStrategy());
		return result;
	}

	public static <T> String toMaskJsonString(T content) {
		return JSON.toJSONString(content, SERIALIZE_CONFIG, MASK_FILTER);
	}

	static class MaskFilter implements ValueFilter {

		@Override
		public Object process(Object object, String name, Object value) {
			if(!isString(value)){
				return value;
			}
			String stringValue = String.class.cast(value);
			Class<?> clazz = object.getClass();
			try {
				// JSONObject Specifies Processing Field
				if(MASK_FIELDS.containsKey(name)){
					MaskStrategy maskStrategy = MASK_FIELDS.get(name);
					return maskStrategy.process(stringValue);
				}
				Field field = clazz.getDeclaredField(name);
				field.setAccessible(true);
				// Java Bean Label@Ignore Annotation
				Ignore ignore = AnnotationUtils.getAnnotation(field, Ignore.class);
				if(ignore != null && value != null) {
					return "***";
				}
				// Java Bean Label@Mask Annotation
				Mask mask = AnnotationUtils.getAnnotation(field, Mask.class);
				if(mask != null && StringUtils.isNotBlank(mask.pattern())) {
					return OutMaskUtil.replaceWithMask(mask.pattern(), stringValue);

				}
			} catch (Exception e) {
				// ignore
			}
			return value;
		}

		private boolean isString(Object value){
			if(value == null) {
				return false;
			}
			return value instanceof String;
		}
	}

}

For Java beans, the original @Ignore and @Mask annotations are also used.For JSONObject s, the fields that need to be processed need to be added manually and a log printing policy can be specified.The policy interfaces are as follows:

public interface MaskStrategy {

	String process(String value);

}

Here, idNo uses the Print empty policy, which means it won't be printed.

public class NullMaskStrategy implements MaskStrategy {

	@Override
	public String process(String value) {
		return null;
	}

}

Log desensitization using JSON eliminates the need to override the toString method of Object objects.

Tags: Java JSON Database Apache

Posted on Wed, 27 May 2020 17:36:58 -0700 by auday1982