{"id":21080,"date":"2019-07-02T14:02:29","date_gmt":"2019-07-02T13:02:29","guid":{"rendered":"https:\/\/www.ch.imperial.ac.uk\/rzepa\/blog\/?p=21080"},"modified":"2023-02-06T12:02:43","modified_gmt":"2023-02-06T12:02:43","slug":"metadata-why","status":"publish","type":"post","link":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080","title":{"rendered":"Metadata. Why?"},"content":{"rendered":"
\n

I have had some interesting discussions recently regarding metadata. What emerges is that it can be quite a broadly defined concept and it is clear that a variety of answers might be obtained when asking the simple question “what is it useful for?” Here I set out some of my answers to that question.<\/p>\n

    \n
  1. Metadata vs<\/em> Data. Questions such as\u00a0where is the continuum between data\/metadata and whether the metadata is fine-grained or more broadly-grained.<\/li>\n
  2. What is its ultimate destination? Should metadata reside inside a complete package or container of data, serving the purpose of succinctly describing what to expect in that package? Or should it reside entirely separately from the data package in some sort of metadata store (MDS)?<\/li>\n
  3. Are there issues of trust or provenance? Thus, how was the metadata created, by a person or a process and when? Has it been changed since it was created? If so, what are the revisions? Does the metadata adhere to a specified structure and has it been\u00a0been validated against that structure.<\/li>\n<\/ol>\n

    Some context needs to be applied before answering such questions (context is perhaps a synonym for metadata!)<\/p>\n

      \n
    1. Firstly, I am going to use metadata here in the context of describing data itself (i.e.<\/em> rather than other research objects such as journal articles). This would include answers to questions such as:\n
        \n
      1. who created both the data and its metadata.<\/li>\n
      2. when were both created and perhaps modified.<\/li>\n
      3. where the data is stored<\/li>\n
      4. what are its defined internal structures (sometimes also called \u00a0MEDIA types).<\/li>\n
      5. who its “publisher” is (the organisation where the data was produced or is curated).<\/li>\n
      6. what are the access and re-use rights associated with the data.<\/li>\n<\/ol>\n

        These are broad-grained provenance if you like.<\/p>\n<\/li>\n

      7. Next, metadata describing the specific the context of the data, e.g.<\/em>\u00a0in my case the chemistry associated with it.\n
          \n
        1. Is it about a molecule?<\/li>\n
        2. if so what is the nature of the molecule?<\/li>\n
        3. Is it computational data about a molecule.<\/li>\n
        4. If so, what software was used for the computations and its parameters, inputs and outputs.<\/li>\n
        5. Might it be instrumental data recorded for a molecule?<\/li>\n
        6. If the latter, does it record the instrument and its settings?<\/li>\n<\/ol>\n

          We are now moving into fine-grained metadata, and perhaps even crossing the boundary into data itself, since the parameters for either software or instruments can be large and complex and are often so heavily mixed into the data itself that their extrication may be a challenge.<\/p>\n<\/li>\n

        7. Finally, what is the purpose of creating and storing such metadata.\n
            \n
          1. Here the context is of “discoverability” (of the data itself) and perhaps also<\/li>\n
          2. “R<\/strong>eusability” and\/or “I<\/strong>nteroperability (of the data itself).<\/li>\n
          3. These attributes are nicely summarised by the acronym F<\/strong>AIR, where discoverability is specified by both\u00a0F<\/strong>indability and A<\/strong>ccessibility.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

            Before introducing examples based on metadata with the focus on discoverability, I want to distinguish between locally packaged metadata and separated metadata (Qu. 2 above). The examples below relate purely to the latter, which has been created as a separate entity by registration with an agency such as DataCite. Such registration also addresses Qu. 3 above about trust. This external agency adds trust by recording the identity of the person (or a process or workflow initiated by a person) registering the metadata together with the registration date (the Datestamp) and also monitors any changes to the metadata (which is allowed) by keeping its version history. Interestingly, there seems to be no mechanism to record any processes or workflows used to create \u00a0metadata so as to learn how the metadata itself was assembled. Nor have I seen much discussion of this aspect; one for the future I fancy.<\/p>\n

            I now introduce some examples of discoverability. The descriptions are quite short and are meant to be used in conjunction with a “reverse-engineering” of the (somewhat) human readable search query. These queries are also deposited as \u00a0“data”, \u00a0at DOI: 10.14469\/hpc\/5920<\/a><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
            Entry<\/th>\nDescription<\/th>\nElasticsearch query<\/th>\n<\/tr>\n
            1<\/td>\nMedia (MIME) type<\/td>\nhttps:\/\/commons.datacite.org\/?query=media.media_type:chemical\/x-mnpub*<\/small><\/a><\/span><\/td>\n<\/tr>\n
            2<\/td>\nCombining Media with the DataCite Subject<\/td>\nhttps:\/\/commons.datacite.org\/?query=media.media_type:chemical\/x-mnpub*+AND+subjects.subjectScheme:inchikey+AND+subjects.subject:XZYDALXOGPZGNV-UHFFFAOYSA-M+AND+media.media_type:chemical\/x-gaussian*<\/small><\/a><\/span><\/td>\n<\/tr>\n
            3<\/td>\nCombining ORCID with Media<\/td>\nhttps:\/\/commons.datacite.org\/?query=contributors.nameIdentifiers.nameIdentifier:*0000-0002-8635-8390+AND+media.media_type:chemical\/x-mnpub*<\/small><\/a><\/span><\/td>\n<\/tr>\n
            4<\/td>\nExploiting Subject<\/td>\nhttps:\/\/commons.datacite.org\/?query=subjects.subjectScheme:Gibbs_Energy+AND+subjects.subject:”-39.946176″<\/small><\/a><\/span><\/td>\n<\/tr>\n
            5<\/td>\nExploiting Subject with range query<\/td>\nhttps:\/\/commons.datacite.org\/?query=subjects.subjectScheme:Gibbs_energy+AND+subjects.subject:[\\-649.1 TO \\-649.8]<\/small><\/a><\/span><\/td>\n<\/tr>\n
            6<\/td>\nNested search with two Subjects<\/td>\nhttps:\/\/commons.datacite.org\/?query=(subjects.subjectScheme:inchikey+AND+subjects.subject:”-1082.980914″)+AND+(subjects.subjectScheme:Gibbs_Energy+AND+subjects.subject:KTOSDSJYNBIDCN-UHFFFAOYSA-N)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            Nested search with two Subjects transposed<\/td>\nhttps:\/\/commons.datacite.org\/?query=(subjects.subjectScheme:inchikey+AND+subjects.subject:KTOSDSJYNBIDCN-UHFFFAOYSA-N)+AND+(subjects.subjectScheme:Gibbs_Energy+AND+subjects.subject:”-1082.980914″)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            7<\/td>\nTwo different Media types<\/td>\nhttps:\/\/commons.datacite.org\/?query=media.media_type:chemical\/x-gaussian*+AND+media.media_type:chemical\/x-mnpub*<\/small><\/a><\/span><\/td>\n<\/tr>\n
            8<\/td>\nLicense type<\/td>\nhttps:\/\/commons.datacite.org\/?query=rightsList.rights:”Creative Commons Public Domain Dedication (CC0 1.0)”<\/small><\/a><\/span><\/td>\n<\/tr>\n
            9<\/td>\nExploiting subjectscheme<\/td>\nhttps:\/\/commons.datacite.org\/?query=media.media_type:chemical\/x-mnpub*+AND+subjects.subjectScheme:NMR_Nucleus+AND+subjects.subject:1H<\/small><\/a><\/span><\/td>\n<\/tr>\n
            10<\/td>\nExploiting subjectscheme<\/td>\nhttps:\/\/commons.datacite.org\/?query=media.media_type:chemical\/x-mnpub*+AND+subjects.subjectScheme:NMR_Pulse+AND+subjects.subject:1D<\/small><\/a><\/span><\/td>\n<\/tr>\n
            11<\/td>\nSimple PID query<\/td>\nhttps:\/\/commons.datacite.org\/?query=identifier:*10.14469\/hpc*<\/small><\/a><\/span><\/td>\n<\/tr>\n
            12<\/td>\nCombining ORCID with PID query<\/td>\nhttps:\/\/commons.datacite.org\/?query=(contributors.nameIdentifiers.nameIdentifier:*0000-0002-8635-8390)+AND+(identifier:*10.14469\/hpc*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            13<\/td>\nCombing researcher name with PID query<\/td>\nhttps:\/\/commons.datacite.org\/?query=(id:10.14469\/hpc*)+AND+(contributors.name:Henry+Rzepa)<\/small><\/a><\/span> <\/p>\n

            \nhttps:\/\/commons.datacite.org\/?query=(id:10.14469\/hpc*)+AND+(creators.name:Henry+Rzepa) <\/small><\/a><\/span> <\/p>\n<\/td>\n<\/tr>\n

            14<\/td>\nEntries in specific repository (Imperial) referencing specific Journal<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:10.1021\/acs.orglett*)+AND+(identifier:*10.14469\/hpc*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            15<\/td>\nEntries in specific repository (Cambridge) referencing specific Journal<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:10.1021\/acs.orglett*)+AND+(identifier:*10.17863\/cam*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            18<\/td>\nEntries in specific repository (Cambridge) referencing all publisher journals<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:10.1021\/acs*)+AND+(identifier:*10.17863\/cam*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            16<\/td>\nEntries in all repositories except one referencing specific Journal<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:10.1021\/acs.orglett*)+NOT+(identifier:*10.5517*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            17<\/td>\nEntries in specific repository referencing one publisher<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:10.1021*)+AND+(identifier:*10.5517*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            19<\/td>\nEntries in all publisher journals, excluding one data repository<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:10.1021*)+NOT+(identifier:*10.5517*)<\/small><\/a><\/span><\/td>\n<\/tr>\n
            20<\/td>\nEntries in Institutional repository referencing datasets<\/td>\nhttps:\/\/commons.datacite.org\/?query=(relatedIdentifiers.relatedIdentifier:*10.14469\/spiral*)+AND+(identifier:*)+AND+(types.resourceTypeGeneral:Dataset)<\/small><\/a><\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

             <\/p>\n

            The examples above reveal a somewhat a not entirely human-friendly syntax; with each of them some effort at “de-bugging” was needed to make them work. I gather from the \u00a0PIDForum<\/a> that a more friendly GUI to achieve this is on their radar. As I develop or discover more examples of such searches I will add them to the list above at DOI: 10.14469\/hpc\/5920<\/a>. Meanwhile, if \u00a0you want to use any of the above as a template for your own searches do please explore.<\/p>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

            I have had some interesting discussions recently regarding metadata. What emerges is that it can be quite a broadly defined concept and it is clear that a variety of answers might be obtained when asking the simple question “what is it useful for?” Here I set out some of my answers to that question. Metadata […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":5,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[2],"tags":[],"ppma_author":[2661],"class_list":["post-21080","post","type-post","status-publish","format-standard","hentry","category-chemical-it"],"yoast_head":"\nMetadata. Why? - Henry Rzepa's Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Metadata. Why? - Henry Rzepa's Blog\" \/>\n<meta property=\"og:description\" content=\"I have had some interesting discussions recently regarding metadata. What emerges is that it can be quite a broadly defined concept and it is clear that a variety of answers might be obtained when asking the simple question “what is it useful for?” Here I set out some of my answers to that question. Metadata […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080\" \/>\n<meta property=\"og:site_name\" content=\"Henry Rzepa's Blog\" \/>\n<meta property=\"article:published_time\" content=\"2019-07-02T13:02:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-06T12:02:43+00:00\" \/>\n<meta name=\"author\" content=\"Henry Rzepa\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Henry Rzepa\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Metadata. Why? - Henry Rzepa's Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080","og_locale":"en_GB","og_type":"article","og_title":"Metadata. Why? - Henry Rzepa's Blog","og_description":"I have had some interesting discussions recently regarding metadata. What emerges is that it can be quite a broadly defined concept and it is clear that a variety of answers might be obtained when asking the simple question “what is it useful for?” Here I set out some of my answers to that question. Metadata […]","og_url":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080","og_site_name":"Henry Rzepa's Blog","article_published_time":"2019-07-02T13:02:29+00:00","article_modified_time":"2023-02-06T12:02:43+00:00","author":"Henry Rzepa","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Henry Rzepa","Estimated reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080#article","isPartOf":{"@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080"},"author":{"name":"Henry Rzepa","@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/#\/schema\/person\/2b40f7b9c872a4dc1547e040a11b6281"},"headline":"Metadata. Why?","datePublished":"2019-07-02T13:02:29+00:00","dateModified":"2023-02-06T12:02:43+00:00","mainEntityOfPage":{"@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080"},"wordCount":1156,"commentCount":2,"articleSection":["Chemical IT"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080","url":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080","name":"Metadata. Why? - Henry Rzepa's Blog","isPartOf":{"@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/#website"},"datePublished":"2019-07-02T13:02:29+00:00","dateModified":"2023-02-06T12:02:43+00:00","author":{"@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/#\/schema\/person\/2b40f7b9c872a4dc1547e040a11b6281"},"breadcrumb":{"@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?p=21080#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog"},{"@type":"ListItem","position":2,"name":"Metadata. Why?"}]},{"@type":"WebSite","@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/#website","url":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/","name":"Henry Rzepa's Blog","description":"Chemistry with a twist","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/#\/schema\/person\/2b40f7b9c872a4dc1547e040a11b6281","name":"Henry Rzepa","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/897b6740f7f599bca7942cdf7d7914af5988937ae0e3869ab09aebb87f26a731?s=96&d=blank&r=g370be3a7397865e4fd161aefeb0a5a85","url":"https:\/\/secure.gravatar.com\/avatar\/897b6740f7f599bca7942cdf7d7914af5988937ae0e3869ab09aebb87f26a731?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/897b6740f7f599bca7942cdf7d7914af5988937ae0e3869ab09aebb87f26a731?s=96&d=blank&r=g","caption":"Henry Rzepa"},"description":"Henry Rzepa is Emeritus Professor of Computational Chemistry at Imperial College London.","sameAs":["https:\/\/orcid.org\/0000-0002-8635-8390"],"url":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/?author=1"}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pDef7-5u0","jetpack-related-posts":[],"jetpack_likes_enabled":false,"authors":[{"term_id":2661,"user_id":1,"is_guest":0,"slug":"admin","display_name":"Henry Rzepa","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/897b6740f7f599bca7942cdf7d7914af5988937ae0e3869ab09aebb87f26a731?s=96&d=blank&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=\/wp\/v2\/posts\/21080","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=21080"}],"version-history":[{"count":20,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=\/wp\/v2\/posts\/21080\/revisions"}],"predecessor-version":[{"id":25915,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=\/wp\/v2\/posts\/21080\/revisions\/25915"}],"wp:attachment":[{"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=21080"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=21080"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=21080"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.ch.ic.ac.uk\/rzepa\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fppma_author&post=21080"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}