Python迭代列表和字典

Question

嗨，我重新提出这个问题，但提供了更多的信息，我正在努力实现。这几天我都快疯了，我好像没什么进展。基本上，我有这样的数据结构：

data_in =\
{'map': {'command_line': u'command goes here',
         'scaninfo': {u'tcp': {'method': u'syn', 'services': u'80,443'}},
         'stats': {'downhosts': u'0',
                   'elapsed': u'1.71',
                   'timestr': u'Thu Mar 20 18:18:09 2014',
                   'totalhosts': u'3',
                   'uphosts': u'3'}},
 'scan': {u'2a00:2384:0:208f::13': {'addresses': {u'ipv6': u'2a00:2384:0:f467::13',
                                                  u'mac': u'00:gf:88:9:56:D5'},
                                    'hostname': u'static.abc.com',
                                    'status': {'reason': u'nd-response',
                                               'state': u'up'},
                                    u'tcp': {80: {'conf': u'3',
                                                  'cpe': '',
                                                  'extrainfo': '',
                                                  'name': u'http',
                                                  'product': '',
                                                  'reason': u'syn-ack',
                                                  'state': u'open',
                                                  'version': ''},
                                             443: {'conf': u'3',
                                                   'cpe': '',
                                                   'extrainfo': '',
                                                   'name': u'https',
                                                   'product': '',
                                                   'reason': u'syn-ack',
                                                   'script': {u'ssl-cert': u'place holder'},
                                                   'state': u'open',
                                                   'version': ''}},
                                    'vendor': {u'00:0C:29:7C:13:D3': u'VMware'}},
          u'2a00:2384:0:208f::15': {'addresses': {u'ipv6': u'a848:2384:0:3456::15',
                                                  u'mac': u'00:gf:29:99:6D:96'},
                                    'hostname': u'static.xyz.com',
                                    'status': {'reason': u'nd-response',
                                               'state': u'up'},
                                    u'tcp': {80: {'conf': u'3',
                                                  'cpe': '',
                                                  'extrainfo': '',
                                                  'name': u'http',
                                                  'product': '',
                                                  'reason': u'syn-ack',
                                                  'state': u'open',
                                                  'version': ''},
                                             443: {'conf': u'3',
                                                   'cpe': '',
                                                   'extrainfo': '',
                                                   'name': u'https',
                                                   'product': '',
                                                   'reason': u'syn-ack',
                                                   'script': {u'ssl-cert': u'place holder'},
                                                   'state': u'open',
                                                   'version': ''}},
                                    'vendor': {u'00:0C:67:99:6f:96': u'VMware'}},
          u'2a00:2384:0:208f::16': {'addresses': {u'ipv6': u'8938:8584:0:8685::16',
                                                  u'mac': u'00:54:29:fg:55:0F'},
                                    'hostname': u'static.edf.com',
                                    'status': {'reason': u'nd-response',
                                               'state': u'up'},
                                    u'tcp': {80: {'conf': u'3',
                                                  'cpe': '',
                                                  'extrainfo': '',
                                                  'name': u'http',
                                                  'product': '',
                                                  'reason': u'syn-ack',
                                                  'state': u'open',
                                                  'version': ''},
                                             443: {'conf': u'3',
                                                   'cpe': '',
                                                   'extrainfo': '',
                                                   'name': u'https',
                                                   'product': '',
                                                   'reason': u'syn-ack',
                                                   'script': {u'ssl-cert': u'place holder'},
                                                   'state': u'open',
                                                   'version': ''}},
                                    'vendor': {u'00:0C:55:AE:33:ff': u'VMware'}}}}

需要创建一个简化版本，如下所示：

data_out =\
[{'address': u'2a00:2384:0:208f::13',
  'hostname': u'static.bt.com',
  'ports': [{80: {'reason': u'syn-ack', 'state': u'open'}},
            {443: {'reason': u'syn-ack',
                   'ssl_cert': u'place   holder',
                   'state': u'open'}}]}]

根据@jornsharpe之前的建议，我已经创建了一个helper函数，使我能够找到键。这证明是有帮助的，但我仍在努力取得预期的结果。你知道吗

def find_key(data, search_key, out=None):
"""Find all values from a nested dictionary for a given key."""
if out is None:
    out = []
if isinstance(data, dict):
    if search_key in data:
        out.append(data[search_key])
    for key in data:
        find_key(data[key], search_key, out)
return out

任何帮助都会非常感激！你知道吗

Answer 1

这并不难；您只需遍历并查看导致所需内容的数据结构—由于格式不好，这变得更加困难，因此我重新缩进了您的输入，并标记了要查找的键（<==）和字段（!!!）：

data_in = {
    'map': {
        'stats': {
            'uphosts': u'3',
            'timestr': u'Thu Mar 20 18:18:09 2014',
            'downhosts': u'0',
            'totalhosts': u'3',
            'elapsed': u'1.71'
        },
        'scaninfo': {
            u'tcp': {
                'services': u'80,443',
                'method': u'syn'
            }
        },
        'command_line': u'command goes here'
    },
    'scan': {                           # <==
        u'2a00:2384:0:208f::13': {          # <== !!!
            'status': {
                'state': u'up',
                'reason': u'nd-response'
            },
            'hostname': u'static.abc.com',      # !!!
            'vendor': {
                u'00:0C:29:7C:13:D3': u'VMware'
            },
            'addresses': {
                u'mac': u'00:gf:88:9:56:D5',
                u'ipv6': u'2a00:2384:0:f467::13'
            },
            u'tcp': {                       # <==
                80: {                       # <== !!!
                    'product': '',
                    'state': u'open',           # !!!
                    'version': '',
                    'name': u'http',
                    'conf': u'3',
                    'extrainfo': '',
                    'reason': u'syn-ack',       # !!!
                    'cpe': ''
                },
                443: {                      # <== !!!
                    'product': '',
                    'state': u'open',           # !!!
                    'version': '',
                    'name': u'https',
                    'conf': u'3',
                    'script': {             # <==
                        u'ssl-cert': u'place holder'  # !!!
                    },
                    'extrainfo': '',
                    'reason': u'syn-ack',       # !!!
                    'cpe': ''
                }
            }
        },
        u'2a00:2384:0:208f::15': {
            'status': {
                'state': u'up',
                'reason': u'nd-response'
            },
            'hostname': u'static.xyz.com',
            'vendor': {
                u'00:0C:67:99:6f:96': u'VMware'
            },
            'addresses': {
                u'mac': u'00:gf:29:99:6D:96',
                u'ipv6': u'a848:2384:0:3456::15'
            },
            u'tcp': {
                80: {
                    'product': '',
                    'state': u'open',
                    'version': '',
                    'name': u'http',
                    'conf': u'3',
                    'extrainfo': '',
                    'reason': u'syn-ack',
                    'cpe': ''
                },
                443: {
                    'product': '',
                    'state': u'open',
                    'version': '',
                    'name': u'https',
                    'conf': u'3',
                    'script': {
                        u'ssl-cert': u'place holder'
                    },
                    'extrainfo': '',
                    'reason': u'syn-ack',
                    'cpe': ''
                }
            }
        },
        u'2a00:2384:0:208f::16': {
            'status': {
                'state': u'up',
                'reason': u'nd-response'
            },
            'hostname': u'static.edf.com',
            'vendor': {
                u'00:0C:55:AE:33:ff': u'VMware'
            },
            'addresses': {
                u'mac': u'00:54:29:fg:55:0F',
                u'ipv6': u'8938:8584:0:8685::16'
            },
            u'tcp': {
                80: {
                    'product': '',
                    'state': u'open',
                    'version': '',
                    'name': u'http',
                    'conf': u'3',
                    'extrainfo': '',
                    'reason': u'syn-ack',
                    'cpe': ''
                },
                443: {
                    'product': '',
                    'state': u'open',
                    'version': '',
                    'name': u'https',
                    'conf': u'3',
                    'script': {
                        u'ssl-cert': u'place holder'
                    },
                    'extrainfo': '',
                    'reason': u'syn-ack',
                    'cpe': ''
                }
            }
        }
    }
}

同样，对于您想要的输出（经过适当的调整）：

data_out = [
    {
        'address': u'2a00:2384:0:208f::13',
        'hostname': u'static.bt.com',
        'ports': {
            80: {
                'state': u'open',
                'reason': u'syn-ack'
            },
            443: {
                'ssl_cert': u'place   holder',
                'state': u'open',
                'reason': u'syn-ack'
            }
        }
    }
]

然后提取变成：

def remap_port(port, port_data):
    result = {
        "state":  port_data["state"],
        "reason": port_data["reason"]
    }
    try:
        result["ssl_cert"] = port_data["script"]["ssl-cert"]
    except KeyError:
        pass
    return port, result

def remap_scanned_address(address, address_data):
    return {
        "address":  address,
        "hostname": address_data["hostname"],
        "ports":    dict(remap_port(port, port_data) for port,port_data in address_data["tcp"].items())
    }

def remap_scan_data(data_in):
    return [remap_scanned_address(address, address_data) for address, address_data in data_in["scan"].items()]

data_out = remap_scan_data(data_in)

从而产生所需的输出

[{'address': u'2a00:2384:0:208f::13',
  'hostname': u'static.abc.com',
  'ports': {80: {'reason': u'syn-ack', 'state': u'open'},
            443: {'reason': u'syn-ack',
                  'ssl_cert': u'place holder',
                  'state': u'open'}}},
 {'address': u'2a00:2384:0:208f::15',
  'hostname': u'static.xyz.com',
  'ports': {80: {'reason': u'syn-ack', 'state': u'open'},
            443: {'reason': u'syn-ack',
                  'ssl_cert': u'place holder',
                  'state': u'open'}}},
 {'address': u'2a00:2384:0:208f::16',
  'hostname': u'static.edf.com',
  'ports': {80: {'reason': u'syn-ack', 'state': u'open'},
            443: {'reason': u'syn-ack',
                  'ssl_cert': u'place holder',
                  'state': u'open'}}}]