Package de.cxp.ocs.smartsuggest

Class S3ArchiveProvider

java.lang.Object
de.cxp.ocs.smartsuggest.spi.CompoundIndexArchiveProvider
de.cxp.ocs.smartsuggest.S3ArchiveProvider
All Implemented Interfaces:
AbstractDataProvider<IndexArchive>, IndexArchiveProvider, Closeable, AutoCloseable
public class S3ArchiveProvider extends CompoundIndexArchiveProvider implements Closeable
Stores the index-archives in a s3 bucket (optionally with some prefix) in the following structure: 
     s3://{bucket}/{prefix}{indexName}/
          - {suffix_1}/archive.tar.gz
          - {suffix_1}/archive.tar.gz
Configuration for this provider:
  • bucket: Required: The name of the s3 bucket
  • prefix: Optional value that should be prepended to all keys. If it does not end with a slash, it won't be added, so the prefix will just be part of the index name prefix.
  • region: Optional value to specify in which region the bucket is located
Make sure the bucket exists prior to the startup of the according service, as verification is done at startup and not at runtime. A bucket won't be created automatically, since this provider won't choose a policy / ACL rules for you.

  • Constructor Details

    • S3ArchiveProvider

      public S3ArchiveProvider()

  • Method Details

    • configure

      public void configure(Map<String,Object> config)
      Description copied from interface: AbstractDataProvider

      This method is always called directly after instantiating the data provider with its no-args-constructor. The given configuration is passed through the QuerySuggestManager and might be empty.

      If this data provider is unusable due to missing configuration, it should throw an Exception, so it will be dropped

      Specified by:
      configure in interface AbstractDataProvider<IndexArchive>
      Parameters:
      config - specific data provider configuration (never null, but may be empty)

    • getIndexSuffixes

      public Collection<String> getIndexSuffixes(String indexName)
      Specified by:
      getIndexSuffixes in class CompoundIndexArchiveProvider

    • store

      public void store(String indexName, IndexArchive archive) throws IOException
      Specified by:
      store in interface IndexArchiveProvider
      Throws:
      IOException

    • getLastDataModTime

      public long getLastDataModTime(String indexName) throws IOException
      Description copied from interface: AbstractDataProvider

      Get the timestamp from when the data was modified the last time. For every change of that timestamp, the data will be pulled and indexed into suggest index.

      Setting the timestamp at the data is optional, but if it is set there, it MUST be the same timestamp, otherwise the data is rejected. This feature is used to avoid potential concurrency issues.

      If data is not available at all, a value < 0 should be returned.

      Specified by:
      getLastDataModTime in interface AbstractDataProvider<IndexArchive>
      Parameters:
      indexName - identifier for the requested data
      Returns:
      unix timestamp in millis
      Throws:
      IOException - if resource is not available

    • hasData

      public boolean hasData(String indexName)
      Description copied from interface: AbstractDataProvider

      Respond with 'true' if this provider is generally able to provide data for the requested index. This is a quick check when initializing the suggesters, so it should not take too long.

      It's also possible to return a static "true" here and do the expensive availability check at the getLastDataModTime method, which is called async. If getLastDataModTime returns a value < 0, data update are canceled as well.

      Specified by:
      hasData in interface AbstractDataProvider<IndexArchive>
      Parameters:
      indexName - identifier for the requested data
      Returns:
      if data is available

    • loadData

      public IndexArchive loadData(String indexName) throws IOException
      Specified by:
      loadData in interface AbstractDataProvider<IndexArchive>
      Parameters:
      indexName - identifier for the requested data
      Returns:
      suggest data
      Throws:
      IOException - if data couldn't be loaded

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable