Class Fetcher

Fetcher

The Fetcher object is a helper object for a quadstore which turns it from an offline store to an online store. The fetcher deals with loading data files rom the web,

  • figuring how to parse them. It will also refresh, remove, the data
  • and put back the data to the web.

Implements

  • CallbackifyInterface

Constructors

Properties

_fetch: Fetch
appNode: NamedNode

Denoting this session

fetchCallbacks: FetchCallbacks

fetchCallbacks[uri].push(callback)

fetchQueue: FetchQueue
fireCallbacks: Function

Methods added by calling Util.callbackify in the constructor

handlers: typeof Handler[]
lookedUp: BooleanMap
mediatypes: MediatypesMap
nonexistent: BooleanMap

Keep track of explicit 404s -> we can overwrite etc

ns: {
    [k: string]: ((ln) => Quad_Predicate);
}

Type declaration

  • [k: string]: ((ln) => Quad_Predicate)
      • (ln): Quad_Predicate
      • Parameters

        • ln: string

        Returns Quad_Predicate

redirectedTo: Record<string, string>

Redirected from key uri to value uri

requested: RequestedMap

this.requested[uri] states: undefined no record of web access or records reset true has been requested, fetch in progress 'done' received, Ok 401 Not logged in 403 HTTP status unauthorized 404 Resource does not exist. Can be created etc. 'redirected' In attempt to counter CORS problems retried. 'parse_error' Parse error 'unsupported_protocol' URI is not a protocol Fetcher can deal with other strings mean various other errors.

store: Store
timeout: number
timeouts: TimeOutsMap

List of timeouts associated with a requested URL

CONTENT_TYPE_BY_EXT: Record<string, string>
HANDLERS: {
    [handlerName: number]: Handler;
}

Type declaration

  • [handlerName: number]: Handler
crossSiteProxyTemplate: any

Methods

  • Parameters

    • uri: string
    • callback: UserCallback

    Returns void

  • Parameters

    • handler: typeof Handler

    Returns void

  • Records a status message (as a literal node) by appending it to the request's metadata status collection.

    Parameters

    • req: BlankNode
    • statusMessage: string

    Returns void

  • Parameters

    • rdfType: NamedNode
    • req: Quad_Subject
    • kb: Store
    • locURI: string

    Returns void

  • Parameters

    • originalUri: string
    • _options: any

      DEPRECATED

    • timeout: number

    Returns void

  • Parameters

    • parentURI: string

      URI of parent container

    • folderName: string

      Optional folder name (slug)

    • data: string

      Optional folder metadata

    Returns Promise<Response>

  • Create an empty resource if it really does not exist Be absolutely sure something does not exist before creating a new empty file as otherwise existing could be deleted.

    Parameters

    • doc: NamedNode

      The resource

    • contentType: "text/turtle" = TurtleContentType
    • data: string = ''

    Returns Promise<ExtendedResponse>

  • Records errors in the system on failure:

    • Adds an entry to the request status collection
    • Adds an error triple with the fail message to the metadata
    • Fires the 'fail' callback
    • Rejects with an error result object, which has a response object if any

    Parameters

    • options: {
          original: Quad_Subject;
          req: BlankNode;
      } & Partial<AutoInitOptions>
    • errorMessage: string
    • statusCode: StatusValues
    • Optional response: ExtendedResponse

    Returns Promise<FetchError>

  • Looks up response header.

    Parameters

    • doc: NamedNode
    • header: string

    Returns undefined | string[]

    a list of header values found in a stored HTTP response, or [] if response was found but no header found, or undefined if no response is available. Looks for { [] link:requestedURI ?uri; link:response [ httph:header-name ?value ] }

  • Parameters

    • uri: string

    Returns undefined | ContentType

  • Parameters

    • iri: string | NamedNode

    Returns void

  • Tests whether a request is being made to a cross-site URI (for purposes of retrying with a proxy)

    Parameters

    • uri: string

    Returns boolean

  • Parameters

    • docuri: string

    Returns boolean

  • Parameters

    • originalUri: NamedNode
    • rel: string
    • uri: string
    • why: Quad_Graph
    • Optional reverse: boolean

    Returns void

  • Promise-based load function

    Loads a web resource or resources into the store.

    A resource may be given as NamedNode object, or as a plain URI. an array of resources will be given, in which they will be fetched in parallel. By default, the HTTP headers are recorded also, in the same store, in a separate graph. This allows code like editable() for example to test things about the resource.

    Type Parameters

    • T extends string | NamedNode | (string | NamedNode)[]

    Parameters

    • uri: T

      {Array|Array|RDFlibNamedNode|string}

    • Optional options: Partial<AutoInitOptions> = {}

      {Object}

    Returns T extends (string | NamedNode)[]
        ? Promise<Response[]>
        : Promise<Response>

  • Looks up something. Looks up all the URIs a things has.

    Parameters

    • term: Quad_Subject

      canonical term for the thing whose URI is to be dereferenced

    • rterm: Quad_Subject

      the resource which referred to this (for tracking bad links)

    Returns Promise<Response> | Promise<Response>[]

  • Note two nodes are now smushed If only one was flagged as looked up, then the new node is looked up again, which will make sure all the URIs are dereferenced

    Parameters

    • was: Quad_Subject
    • now: Quad_Subject

    Returns void

  • Asks for a doc to be loaded if necessary then calls back

    Calling methods: nowOrWhenFetched (uri, userCallback) nowOrWhenFetched (uri, options, userCallback) nowOrWhenFetched (uri, referringTerm, userCallback, options) <-- old nowOrWhenFetched (uri, referringTerm, userCallback) <-- old

    Options include: referringTerm The document in which this link was found. this is valuable when finding the source of bad URIs force boolean. Never mind whether you have tried before, load this from scratch. forceContentType Override the incoming header to force the data to be treated as this content-type.

    Callback function takes:

    ok True if the fetch worked, and got a 200 response. False if any error happened

    errmessage Text error message if not OK.

    response The fetch Response object (was: XHR) if there was was one includes response.status as the HTTP status if any.

    Parameters

    • uriIn: string | NamedNode
    • Optional p2: Partial<AutoInitOptions> | UserCallback
    • Optional userCallback: UserCallback
    • options: Partial<AutoInitOptions> = {}

    Returns void

  • Parameters

    • linkHeader: string
    • originalUri: NamedNode
    • reqNode: Quad_Graph

    Returns void

  • Parameters

    Returns Promise<Response>

  • Writes back to the web what we have in the store for this uri

    Parameters

    Returns Promise<Response>

  • Parameters

    • term: NamedNode
    • Optional userCallback: UserCallback

    Returns void

  • Parameters

    • term: NamedNode
    • userCallback: UserCallback

    Returns void

  • Parameters

    • docuri: string
    • options: any

    Returns Promise<Response>

  • Parameters

    • response: Response
    • options: {
          req: BlankNode;
          resource: Quad_Subject;
      } & Partial<AutoInitOptions>

    Returns BlankNode

  • A generic web operation, at the fetch() level. does not involve the quad store.

    Returns promise of Response If data is returned, copies it to response.responseText before returning

    Parameters

    • method: HTTPMethods
    • uriIn: string | NamedNode
    • options: Partial<AutoInitOptions> = {}

    Returns Promise<ExtendedResponse>

  • Parameters

    • uri: string

    Returns string

  • Decide on credentials using old XXHR api or new fetch() one

    Parameters

    Returns void

  • Tests whether the uri's protocol is supported by the Fetcher.

    Parameters

    • uri: string

    Returns boolean