Subreddit for posting questions and asking for general advice about all topics related to learning python.
Comparing Two JSON Files For Differences
I need to compare two JSON files that contain a list of dictionaries whose basic format is:
[{"protocol": "S", "type": "", "network": "0.0.0.0", "mask": "0", "distance": "254", "metric": "0", "nexthop_ip": "192.168.122.1", "nexthop_if": "", "uptime": ""}, {"protocol": "O", "type": "", "network": "10.129.30.0", "mask": "24", "distance": "110", "metric": "2", "nexthop_ip": "172.20.10.1", "nexthop_if": "GigabitEthernet0/1", "uptime": "08:58:25"}]Though there are many, many more dictionary items in the list than that shown above. I am not quite sure how best to go about comparing the files to spot differences and return or save those difference to another file, preferably in JSON format, though a CSV would be fine too.
The one gotcha that there may be is I need to exclude, at a minimum, the uptime value as it is constantly changing so it will of course trigger anything looking for changes. Can anyone help get me started please?
Top 1% Commenter
For example:
data1 = [{"protocol": "S", "type": "", "network": "0.0.0.0", "mask": "0", "distance": "254", "metric": "0", "nexthop_ip": "192.168.122.1", "nexthop_if": "", "uptime": ""}, {"protocol": "O", "type": "", "network": "10.129.30.0", "mask": "24", "distance": "110", "metric": "2", "nexthop_ip": "172.20.10.1", "nexthop_if": "GigabitEthernet0/1", "uptime": "08:58:25"}]
data2 = [{"protocol": "S", "type": "", "network": "0.0.0.0", "mask": "0", "distance": "24", "metric": "0", "nexthop_ip": "192.168.122.1", "nexthop_if": "", "uptime": ""}, {"protocol": "O", "type": "", "network": "10.129.30.0", "mask": "24", "distance": "110", "metric": "2", "nexthop_ip": "172.20.10.1", "nexthop_if": "GigabitEthernet0/1", "uptime": "08:58:25"}]
def data_massage(data):
del data['uptime']
return frozenset(data.items())
data1_set = {data_massage(x) for x in data1}
data2_set = {data_massage(x) for x in data2}
# I'm assuming by "compare" you mean you want to know what values are in data2 that were not in data1
for item in data2_set - data1_set:
print(dict(item))
Wow! Thank you. Let me run that.
Few words... Pandas, dataframe...
I had in the back of my head someone would probably suggest that. I have never used pandas, so I wouldn't have any idea how to start, but I suppose Google would, LOL
Top 1% Commenter
no kill like overkill, right?
Top 1% Commenter
Remove the uptime value, convert what's left to frozensets and use a set difference operation.
I'd have to find a way to remove that key,value post processing. It's part of the data that comes in from the network device, I can't, as far as I know, prevent its inclusion, its raw text output formatted by TextFSM.
You can also use unittest. The assertequal method can compare dictionaries and lists of dictionaries and show differences.
Doesn't assert only work if you use debug mode?
I’d use the “sure” package. It’s meant for unit testing, but it does an excellent job of comparing data structures