使用lxm将XML片段插入XML文档

2024-05-29 08:14:52 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一组要合并在一起的XML文件。主XML文档是一个完整的iso19139xml文档,另外两个XML文件可能包含<gmd:descriptiveKeywords>元素。我需要从片段文件中提取这些<gmd:descriptiveKeywords>元素中的任何一个并添加到主文件中。有数百个这样的文件集,所以我需要做一些匹配,以确保我合并的数据集是正确的。在

片段XML文件可能如下所示:

<?xml version="1.0" encoding="UTF-8"?>
<ValueSupplyChain xmlns:gmd="http://www.isotc211.org/2005/gmd"
    xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmx="http://www.isotc211.org/2005/gmx"
    xmlns:xlink="http://www.w3.org/1999/xlink" id="MICA_B1v-101"
    title="MINERALS4EU-EU MINERALS KNOWLEDGE DATA PLATFORM (EU-MKDP)">
    <gmd:descriptiveKeywords>
        <gmd:MD_Keywords id="exploration">
            <gmd:keyword>
                <gco:CharacterString>Exploration</gco:CharacterString>
            </gmd:keyword>
            <gmd:thesaurusName>
                <gmd:CI_Citation>
                    <gmd:title>
                        <gco:CharacterString>MICA ontology
                            (ValueSupplyChainScheme)</gco:CharacterString>
                    </gmd:title>
                    <gmd:date gco:nilReason="unknown"/>
                    <gmd:edition>
                        <gco:CharacterString>2</gco:CharacterString>
                    </gmd:edition>
                    <gmd:identifier>
                        <gmd:MD_Identifier>
                            <gmd:code>
                                <gmx:Anchor
                                    xlink:href="https://w3id.org/mica/ontology/MicaOntology/7418a9ae1cd44847889c2c92408e1e71"
                                />
                            </gmd:code>
                        </gmd:MD_Identifier>
                    </gmd:identifier>
                </gmd:CI_Citation>
            </gmd:thesaurusName>
        </gmd:MD_Keywords>
    </gmd:descriptiveKeywords>
</ValueSupplyChain>

主XML具有如下结构(使用图像,因为XML可能会变得非常大):

ISO XML document structure

理想情况下,我希望在现有关键字部分下面附加相关的片段部分,并创建一个新的主控文档。在

我的问题是,尽管我似乎能够匹配正确的数据集,并找到相关的部分,但我认为我所做的更改永远不会写入到输出目标文件中。在

我的代码是:

^{pr2}$

调试输出如下:

DEBUG:root:T title: BGR BOREHOLE MAP
DEBUG:root:T file has the following number of gmd:descriptiveKeywords sections: 7
DEBUG:root:D title: BGR BOREHOLE MAP
DEBUG:root:T and D titles are the same, we can continue...
DEBUG:root:D file has the following number of gmd:descriptiveKeywords sections: 5
DEBUG:root:Subtotal: 12
DEBUG:root:V title: BGR BOREHOLE MAP
DEBUG:root:T and V titles are the same, we can continue...
DEBUG:root:V file has the following number of gmd:descriptiveKeywords sections: 1
DEBUG:root:Subtotal: 13
DEBUG:root:T title: 3D, 4D AND PREDICTIVE MODELLING OF MAJOR MINERAL BELTS IN EUROPE
DEBUG:root:T file has the following number of gmd:descriptiveKeywords sections: 36
DEBUG:root:D title: 3D, 4D AND PREDICTIVE MODELLING OF MAJOR MINERAL BELTS IN EUROPE
DEBUG:root:T and D titles are the same, we can continue...
DEBUG:root:D file has the following number of gmd:descriptiveKeywords sections: 5
DEBUG:root:Subtotal: 41

从调试信息看来,我成功地添加到gmd:descriptiveKeywords元素,那么列表长度按预期增加,但正如我说的,当我写出XML时,我得到了原始主文件的内容。在

我也尝试过使用ElementTree,但是我遇到了同样的问题;而且输出不支持主目录中使用的名称空间前缀。在

我做错什么了?在

编辑

重现该问题的最小代码如下:

from lxml import etree as et

# Open the master file, which is a well-formed and schema valid ISO 19139 XML record
tree = et.parse('T1_0.xml')
root = tree.getroot()

ns_all = {'gmd': 'http://www.isotc211.org/2005/gmd',
          'gco': 'http://www.isotc211.org/2005/gco',
          'gmx': 'http://www.isotc211.org/2005/gmx',
          'xsi': 'http://www.w3.org/2001/XMLSchema-instance',
          'gml': 'http://www.opengis.net/gml',
          'xlink': 'http://www.w3.org/1999/xlink',
          'geonet': 'http://www.fao.org/geonetwork'}

keywordList = root.findall('gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords', ns_all)

# Just a quick check that everything works as expected
print(len(keywordList)) # Should return 7 for the master file

# Open a well-formed XML file containing content we wish to add to the (or a copy of the) master record
dTree = et.parse('D1_0.xml')
dRoot = dTree.getroot()
DKeywords = dRoot.findall('gmd:descriptiveKeywords', ns_all)

# Just a quick check that everything works as expected
print(len(DKeywords)) # Should return 5 for the D file

# Add the keywords from the second file to the keywords of the master file
keywordList.extend(DKeywords)

# We've added 5 records so the result should be 12
print(len(keywordList)) # I get 12 here

# Write out the new file
tree.write('combinedTD1_0.xml')

# If all worked as expected the new file should have 12
ctree = et.parse('combinedTD1_0.xml')
croot = ctree.getroot()
CKeywords = croot.findall('gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords', ns_all)

print(len(CKeywords)) # I get 7 :(

文件包括:

主文件示例:T1_0.xml

片段文件示例:D1_0.xml

片段文件示例:V1_0.xml


Tags: 文件oftheorgdebughttptitlewww
2条回答

{cda>只向列表添加元素。此操作不会对XML树执行任何操作。在

要将附加的descriptiveKeywords节点作为主控文档中的同级节点插入,可以执行以下操作:

# Get the last of the descriptiveKeywords nodes in the master document
last_kw = keywordList[-1]
# Get the node's parent and its position (index) within the parent
kw_parent = last_kw.getparent()
ix = kw_parent.index(last_kw)

# Insert the descriptiveKeyword nodes from the fragment file as successive siblings
for dk in DKeywords:
    kw_parent.insert(ix+1, dk)
    ix += 1

正如我多次回答的那样,考虑一下XSLT,这是一种专门用来转换XML文件的语言,当需要操作XML文件时,比如合并文档。Python的lxml模块可以运行xslt1.0脚本。在

具体地说,XSLT维护document()函数,您可以通过一个文件名参数将片段节点附加到现有的主节点。另外,XSLT使用Identity Transform复制整个文档,就像使用Muenchian Grouping按不同的关键字对文档进行索引一样。使用这种方法,唯一需要的for循环是遍历文件。在

因为OP没有设置一个可复制的示例,下面是一个使用StackOverflow的前3个用户在python和{a5}标记中的示例演示。主文件从前1位开始。然后,Python脚本迭代以按<tag1>追加第2列,然后再追加第3列:

掌握XML(排名前1的用户)

<?xml version="1.0"?>
<stackoverflow>
  <group lang="python">
    <topusers>
      <user>Martijn Pieters</user>
      <link>https://stackoverflow.com/users/100297/martijn-pieters</link>
      <location>Cambridge, United Kingdom </location>
      <year_rep>70,404</year_rep>
      <total_rep>590,309</total_rep>
      <tag1>python</tag1>
      <tag2>python-3.x</tag2>
      <tag3>python-2.7</tag3>
    </topusers>
  </group>
  <group lang="xslt">
    <topusers>
      <user>Dimitre Novatchev</user>
      <link>https://stackoverflow.com/users/36305/dimitre-novatchev</link>
      <location>United States</location>
      <year_rep>9,922</year_rep>
      <total_rep>197,245</total_rep>
      <tag1>xslt</tag1>
      <tag2>xml</tag2>
      <tag3>xpath</tag3>
    </topusers>
  </group>
</stackoverflow>

排名2的XML(即片段)

^{pr2}$

排名3的XML(即片段)

<?xml version="1.0" encoding="utf-8"?>
<stackoverflow>  
 <group lang="python">
     <topusers>
        <user>unutbu</user>
        <link>https://stackoverflow.com/users/190597/unutbu</link>
        <location></location>
        <year_rep>55,492</year_rep>
        <total_rep>453,267</total_rep>
        <tag1>python</tag1>
        <tag2>pandas</tag2>
        <tag3>numpy</tag3>
      </topusers>    
  </group>
 <group lang="xslt">
      <topusers>
        <user>michael.hor257k</user>
        <link>https://stackoverflow.com/users/3016153/michael-hor257k</link>
        <location></location>
        <year_rep>11,339</year_rep>
        <total_rep>70,473</total_rep>
        <tag1>xslt</tag1>
        <tag2>xml</tag2>
        <tag3>xslt-1.0</tag3>
      </topusers>  
  </group>
</stackoverflow>

XSLT(另存为.xsl文件,与.xml文件位于同一目录中)

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output indent="yes" omit_xml_declaration="no"/>
  <xsl:strip-space elements="*"/>

  <xsl:key name="keyid" match="topusers" use="tag1" />  
  <xsl:param name="fragment" /> 

  <!  IDENTITY TRANSFORM  >
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!  IMPORT XML FRAGMENT  >
  <xsl:template match="group">     
    <xsl:copy>      
      <xsl:copy-of select="@*"/> 
      <xsl:apply-templates select="topusers[generate-id() = generate-id(key('keyid', tag1))]"/>  
    </xsl:copy>
  </xsl:template>

  <!  COPY EXISTING topusers AND APPEND EXTERNAL topusers BY SAME KEYWORD  >
  <xsl:template match="topusers">     
    <xsl:variable select="tag1" name="keyword"/>
    <xsl:for-each select="key('keyid', tag1)">
        <xsl:copy-of select="."/>
    </xsl:for-each>
    <!  PASS PYTHON PARAM INTO document()  >
    <xsl:copy-of select="document($fragment)/stackoverflow/group/topusers[tag1=$keyword]"/>
  </xsl:template>

</xsl:stylesheet>

Python(解析所有xml和xsl文件)

import os
import lxml.etree as et

# CURRENT DIRECTORY OF SCRIPT
cd = os.path.dirname(os.path.abspath(__file__))
master = os.path.join(cd, 'Master.xml')

# LOAD XSL SCRIPT
xsl = et.parse(os.path.join(cd, 'XSLTScript.xsl'))
transform = et.XSLT(xsl)

# ITERATE THROUGH FRAGMENT XML FILES IN DIRECTORY
for f in sorted(os.listdir(cd)):
    if f.endswith('.xml'):
        # LOAD MASTER XML
        doc = et.parse(master)

        print(f)
        # PASS FILE NAME AS PARAMETER FOR XSLT's document()
        n = et.XSLT.strparam(f)
        result = transform(doc, fragment=n)

        # UPDATE MASTER XML
        with open(master, 'wb') as s:
            s.write(result)

输出(每个标签排名前3)

<?xml version="1.0"?>
<stackoverflow>
  <group lang="python">
    <topusers>
      <user>Martijn Pieters</user>
      <link>https://stackoverflow.com/users/100297/martijn-pieters</link>
      <location>Cambridge, United Kingdom </location>
      <year_rep>70,404</year_rep>
      <total_rep>590,309</total_rep>
      <tag1>python</tag1>
      <tag2>python-3.x</tag2>
      <tag3>python-2.7</tag3>
    </topusers>
    <topusers>
      <user>Alex Martelli</user>
      <link>https://stackoverflow.com/users/95810/alex-martelli</link>
      <location>Sunnyvale, CA</location>
      <year_rep>49,172</year_rep>
      <total_rep>540,372</total_rep>
      <tag1>python</tag1>
      <tag2>list</tag2>
      <tag3>c++</tag3>
    </topusers>
    <topusers>
      <user>unutbu</user>
      <link>https://stackoverflow.com/users/190597/unutbu</link>
      <location/>
      <year_rep>55,492</year_rep>
      <total_rep>453,267</total_rep>
      <tag1>python</tag1>
      <tag2>pandas</tag2>
      <tag3>numpy</tag3>
    </topusers>
  </group>
  <group lang="xslt">
    <topusers>
      <user>Dimitre Novatchev</user>
      <link>https://stackoverflow.com/users/36305/dimitre-novatchev</link>
      <location>United States</location>
      <year_rep>9,922</year_rep>
      <total_rep>197,245</total_rep>
      <tag1>xslt</tag1>
      <tag2>xml</tag2>
      <tag3>xpath</tag3>
    </topusers>
    <topusers>
      <user>Martin Honnen</user>
      <link>https://stackoverflow.com/users/252228/martin-honnen</link>
      <location>Germany</location>
      <year_rep>10,046</year_rep>
      <total_rep>92,604</total_rep>
      <tag1>xslt</tag1>
      <tag2>xml</tag2>
      <tag3>xpath</tag3>
    </topusers>
    <topusers>
      <user>michael.hor257k</user>
      <link>https://stackoverflow.com/users/3016153/michael-hor257k</link>
      <location/>
      <year_rep>11,339</year_rep>
      <total_rep>70,473</total_rep>
      <tag1>xslt</tag1>
      <tag2>xml</tag2>
      <tag3>xslt-1.0</tag3>
    </topusers>
  </group>
</stackoverflow>

操作XSLT

与OP的实际主文件和片段文件相匹配的相应XSLT可能看起来像这个未经测试的版本。下面假设关键字与发布的fragement在同一布局中(由于图像关闭了<gmd:descriptiveKeywords>节点,因此无法判断):

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                              xmlns:gmd="http://www.isotc211.org/2005/gmd"
                              xmlns:gco="http://www.isotc211.org/2005/gco"
                              xmlns:gmx="http://www.isotc211.org/2005/gmx"
                              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                              xmlns:gml="http://www.opengis.net/gml"
                              xmlns:xlink="http://www.w3.org/1999/xlink"
                              xmlns:geonet="http://www.fao.org/geonetwork">

  <xsl:output indent="yes" omit_xml_declaration="no"/>
  <xsl:strip-space elements="*"/>

  <xsl:key name="keyid" match="gmd:MD_Keywords" use="gmd:keyword/gco:CharacterString" />  
  <xsl:param name="fragment" /> 

  <!  IDENTITY TRANSFORM  >
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!  IMPORT XML FRAGMENT  >
  <xsl:template match="gmd:descriptiveKeywords">     
    <xsl:copy>      
      <xsl:apply-templates select="gmd:MD_Keywords[generate-id() = generate-id(key('keyid', gmd:keyword/gco:CharacterString))]"/>  
    </xsl:copy>
  </xsl:template>

  <!  COPY EXISTING gmd:MD_Keywords AND APPEND EXTERNAL gmd:MD_Keywords BY SAME KEYWORD  >
  <xsl:template match="gmd:MD_Keywords">     
    <xsl:variable select="gmd:keyword/gco:CharacterString" name="keyword"/>
    <xsl:for-each select="key('keyid', gmd:keyword/gco:CharacterString)">
        <xsl:copy-of select="."/>
    </xsl:for-each>
    <!  PASS PYTHON PARAM INTO document()  >
    <xsl:copy-of select="document($fragment)/ValueSupplyChain/gmd:descriptiveKeywords/gmd:MD_Keywords[gmd:keyword/gco:CharacterString=$keyword]"/>
  </xsl:template>

</xsl:stylesheet>

相关问题 更多 >

    热门问题