Package org.ametys.cms.content.indexing.solr

Class SolrIndexer

    • Method Detail

      • dateFormat

        public static SimpleDateFormat dateFormat()
        Returns the formatter for indexing dates. This is used for adding a dates as formatted strings (and not with date object directly) to prevent indexing of the wrong value because of time zone
        Returns:
        The date format for indexing dates

      • truncateUtf8StringValue

        public static String truncateUtf8StringValue​(String value,
                                             org.slf4j.Logger logger,
                                             String documentId,
                                             String fieldName)
        Truncates (if needed) the given string in order to be indexed without immense term error by Solr. Only the 32766 first bytes of the String will be kept.
        Parameters:
        value - The string value to index
        logger - The logger for logging in WARN level in case the given string is too long and will be truncated. Can be null if you do not want to log.
        documentId - The id of the document being indexed. Can be null if you do not want to log.
        fieldName - The name of the field being indexed. Can be null if you do not want to log.
        Returns:
        The given string value, or its truncation if it is too long (greater than 32766 bytes)

      • _getAutoCommitSolrClient

        protected org.apache.solr.client.solrj.SolrClient _getAutoCommitSolrClient​(String workspaceName)
        Gets the 'autocommit' Solr client
        Parameters:
        workspaceName - The name of the workspace
        Returns:
        the Solr client

      • _getNoAutoCommitSolrClient

        protected org.apache.solr.client.solrj.SolrClient _getNoAutoCommitSolrClient​(String workspaceName)
        Gets the 'no autocommit' Solr client
        Parameters:
        workspaceName - The name of the workspace
        Returns:
        the Solr client

      • _defaultSolrClient

        private org.apache.solr.client.solrj.SolrClient _defaultSolrClient()

      • getCoreNames

        public Set<String> getCoreNames()
                         throws IOException,
                                org.apache.solr.client.solrj.SolrServerException
        Get the names of the Solr cores.
        Returns:
        The names of the Solr cores.
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.

      • getRealCoreNames

        protected Set<String> getRealCoreNames()
                                throws IOException,
                                       org.apache.solr.client.solrj.SolrServerException
        Get the names of the Solr cores.
        Returns:
        The names of the Solr cores.
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.

      • createCore

        public void createCore​(String name)
                throws IOException,
                       org.apache.solr.client.solrj.SolrServerException
        Create a Solr core.
        Parameters:
        name - The name of the core to create.
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.

      • deleteCore

        public void deleteCore​(String name)
                throws IOException,
                       org.apache.solr.client.solrj.SolrServerException
        Delete a Solr core.
        Parameters:
        name - The name of the core to delete.
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.

      • sendSchema

        public void sendSchema()
                throws IOException,
                       org.apache.solr.client.solrj.SolrServerException
        Send the schema.
        Throws:
        IOException - If a communication error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a solr error occurs.

      • addStaticSchemaUpdates

        protected void addStaticSchemaUpdates​(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates,
                                      org.apache.solr.client.solrj.response.schema.SchemaRepresentation staticSchema,
                                      SchemaFields schemaFields)
        Compute the list of SchemaRequest.Update directives from the static schema.
        Parameters:
        updates - The list of SchemaRequest.Update directives to fill.
        staticSchema - The static schema representation.
        schemaFields - The current schema fields, used to track the existing fields (to be filled).

      • addCustomUpdates

        protected void addCustomUpdates​(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates,
                                SchemaFields schemaFields)
        Compute the list of custom SchemaRequest.Update directives.
        Parameters:
        updates - The list of SchemaRequest.Update directives to fill.
        schemaFields - The current schema fields, used to track the existing fields (to be filled).

      • reorderUpdates

        protected void reorderUpdates​(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates)
        Reorder the list of SchemaRequest.Update directives.
        Parameters:
        updates - The list of SchemaRequest.Update directives to reorder.

      • definitionExists

        protected boolean definitionExists​(SchemaDefinition definition,
                                   SchemaFields schemaFields)
        Test if the given schema definition exists in the given SchemaFields reference.
        Parameters:
        definition - The schema definition to test.
        schemaFields - The current schema fields.
        Returns:
        true if the SchemaFields contain the schema definition.

      • clearSchema

        protected void clearSchema​(org.apache.solr.client.solrj.SolrClient solrClient,
                           String collection)
                    throws IOException,
                           org.apache.solr.client.solrj.SolrServerException
        Delete all the fields of the existing schema in the given collection.
        Parameters:
        solrClient - The Solr client
        collection - The collection.
        Throws:
        IOException - If a communication error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a solr error occurs.

      • getSchemaUpdate

        protected org.apache.solr.client.solrj.request.schema.SchemaRequest.Update getSchemaUpdate​(SchemaDefinition definition)
        Get the schema SchemaRequest.Update directive from the given schema definition.
        Parameters:
        definition - The schema definition to add.
        Returns:
        The add SchemaRequest.Update directive.

      • reloadCores

        protected void reloadCores()
                    throws IOException,
                           org.apache.solr.client.solrj.SolrServerException
        Reload the solr cores.
        Throws:
        IOException - If a communication error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a solr error occurs.

      • reloadAclCache

        public void reloadAclCache()
                    throws IOException,
                           org.apache.solr.client.solrj.SolrServerException,
                           RepositoryException
        Reloads the ACL Solr cache for all users
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.
        RepositoryException - If a repository exception occurs when retrieving workspaces.

      • reloadAclCache

        public void reloadAclCache​(String workspaceName)
                    throws IOException,
                           org.apache.solr.client.solrj.SolrServerException
        Reloads the ACL Solr cache for all users
        Parameters:
        workspaceName - The workspace name
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.

      • reloadAclCache

        public void reloadAclCache​(String workspaceName,
                           boolean checkIfNecessary)
                    throws IOException,
                           org.apache.solr.client.solrj.SolrServerException
        Reloads the ACL Solr cache for all users
        Parameters:
        workspaceName - The workspace name
        checkIfNecessary - true to check if the reload is necessary for each segment (i.e. reload only the segments not already in cache)
        Throws:
        IOException - If an I/O error occurs.
        org.apache.solr.client.solrj.SolrServerException - If a Solr error occurs.

      • indexAllContents

        public Map<String,​Object> indexAllContents​(String workspaceName,
                                                 boolean indexAttachments,
                                                 org.apache.solr.client.solrj.SolrClient solrClient)
                                          throws Exception
        Index all the contents in a given workspace.
        Parameters:
        workspaceName - the workspace where to index
        indexAttachments - to index content attachments
        solrClient - The solr client to use
        Returns:
        The indexation result as a Map.
        Throws:
        Exception - if an error occurs while indexing.

      • unindexAllContents

        protected void unindexAllContents​(String workspaceName,
                                  boolean unindexAttachments,
                                  org.apache.solr.client.solrj.SolrClient solrClient)
                           throws Exception
        Unindex all content documents.
        Parameters:
        workspaceName - The workspace name
        unindexAttachments - also unindex content attachments
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while unindexing.

      • indexSubcontents

        public void indexSubcontents​(String collectionId,
                             boolean indexAttachments)
                      throws Exception
        Add or update the child contents of a AmetysObjectCollection into Solr index, for all workspaces and commit
        Parameters:
        collectionId - The id of collection
        indexAttachments - to index content attachments
        Throws:
        Exception - if an error occurs while indexing.

      • indexSubcontents

        public void indexSubcontents​(String collectionId,
                             String workspaceName,
                             boolean indexAttachments)
                      throws Exception
        Index the child contents of a AmetysObjectCollection
        Parameters:
        collectionId - The id of collection
        workspaceName - the workspace where to index
        indexAttachments - to index content attachments
        Throws:
        Exception - if an error occurs while unindexing.

      • indexContent

        public void indexContent​(String contentId,
                         boolean indexAttachments)
                  throws Exception
        Add or update a content into Solr index on all workspaces and commit
        Parameters:
        contentId - The id of the content to index
        indexAttachments - to index content attachments
        Throws:
        Exception - if an error occurs while indexing.

      • indexContent

        public void indexContent​(String contentId,
                         String workspaceName,
                         boolean indexAttachments)
                  throws Exception
        Add or update a content into Solr index
        Parameters:
        contentId - The id of the content to index
        workspaceName - the workspace where to index
        indexAttachments - to index content attachments
        Throws:
        Exception - if an error occurs while indexing.

      • indexContent

        public void indexContent​(String contentId,
                         String workspaceName,
                         boolean indexAttachments,
                         org.apache.solr.client.solrj.SolrClient solrClient)
                  throws Exception
        Add or update a content into Solr index
        Parameters:
        contentId - The id of the content to index
        workspaceName - the workspace where to index
        indexAttachments - to index content attachments
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • indexContents

        public SolrIndexer.IndexationResult indexContents​(Iterable<Content> contents,
                                                  String workspaceName,
                                                  boolean indexAttachments,
                                                  org.apache.solr.client.solrj.SolrClient solrClient)
                                           throws Exception
        Send a collection of contents for indexation in the solr server.
        Parameters:
        contents - the collection of contents to index.
        workspaceName - the workspace where to index
        indexAttachments - to index content attachments
        solrClient - The solr client to use
        Returns:
        the indexation result.
        Throws:
        Exception - if an error occurs while indexing.

      • doIndexContents

        protected SolrIndexer.IndexationResult doIndexContents​(Iterable<Content> contents,
                                                       String workspaceName,
                                                       boolean indexAttachments,
                                                       org.apache.solr.client.solrj.SolrClient solrClient)
                                                throws Exception
        Send some contents for indexation in the solr server.
        Parameters:
        contents - the contents to index.
        workspaceName - The workspace name
        indexAttachments - to index content attachments
        solrClient - The solr client to use
        Returns:
        the indexation result.
        Throws:
        Exception - if an error occurs committing the results.

      • updateSystemProperty

        public void updateSystemProperty​(Content content,
                                 String propertyId,
                                 String workspaceName)
                          throws Exception
        Update the value of a specific system property in a content document.
        Parameters:
        content - The content to update.
        propertyId - The system property ID.
        workspaceName - The workspace name
        Throws:
        Exception - if an error occurs while indexing.

      • updateSystemProperty

        public void updateSystemProperty​(Content content,
                                 String propertyId,
                                 String workspaceName,
                                 org.apache.solr.client.solrj.SolrClient solrClient)
                          throws Exception
        Update the value of a specific system property in a content document.
        Parameters:
        content - The content to update.
        propertyId - The system property ID.
        workspaceName - The workspace name
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • unindexContent

        public void unindexContent​(String contentId,
                           boolean unindexAttachments)
                    throws Exception
        Remove a content from Solr index for all workspaces and commit
        Parameters:
        contentId - The id of content to unindex
        unindexAttachments - also unindex content attachments
        Throws:
        Exception - if an error occurs while indexing.

      • unindexContent

        public void unindexContent​(String contentId,
                           String workspaceName,
                           boolean unindexAttachments)
                    throws Exception
        Remove a content from Solr index
        Parameters:
        contentId - The id of content to unindex
        workspaceName - The workspace where to work in
        unindexAttachments - also unindex content attachments
        Throws:
        Exception - if an error occurs while indexing.

      • unindexContent

        public void unindexContent​(String contentId,
                           String workspaceName,
                           boolean unindexAttachments,
                           org.apache.solr.client.solrj.SolrClient solrClient)
                    throws Exception
        Remove a content from Solr index
        Parameters:
        contentId - The id of content to unindex
        workspaceName - The workspace where to work in
        unindexAttachments - also unindex content attachments
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • unindexContents

        public void unindexContents​(Collection<String> contentIds)
                     throws Exception
        Remove a content from Solr index for all workspaces and commit
        Parameters:
        contentIds - The id of content to unindex
        Throws:
        Exception - if an error occurs while indexing.

      • unindexContents

        public void unindexContents​(Collection<String> contentIds,
                            String workspaceName)
                     throws Exception
        Remove a content from Solr index
        Parameters:
        contentIds - The id of content to unindex
        workspaceName - The workspace where to work in
        Throws:
        Exception - if an error occurs while indexing.

      • doIndexContent

        protected void doIndexContent​(Content content,
                              String workspaceName,
                              org.apache.solr.client.solrj.SolrClient solrClient)
                       throws Exception
        Add or update a content into Solr index
        Parameters:
        content - The content to index
        workspaceName - The workspace where to index
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • indexAclInitValues

        public void indexAclInitValues​(AmetysObject ametysObject,
                               org.apache.solr.common.SolrInputDocument document)
        Indexes read-ACl initial values for object
        Parameters:
        ametysObject - The object
        document - The Solr document

      • _addField

        private <T> void _addField​(org.apache.solr.common.SolrInputDocument document,
                           String fieldName,
                           Set<T> values,
                           Function<T,​String> stringifier)

      • doIndexContentWorkflow

        protected void doIndexContentWorkflow​(Content content,
                                      String workspaceName,
                                      org.apache.solr.client.solrj.SolrClient solrClient)
                               throws Exception
        Index the whole workflow of a content.
        Parameters:
        content - The content.
        workspaceName - The workspace name
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • indexContentAttachments

        public void indexContentAttachments​(ResourceCollection collection,
                                    Content content)
                             throws Exception
        Index content attachments as new entries in the idnex
        Parameters:
        collection - the collection of attachments
        content - the content whose attachments will be indexed
        Throws:
        Exception - if something goes wrong when indexing the attachments of the content

      • indexContentAttachments

        public void indexContentAttachments​(ResourceCollection collection,
                                    Content content,
                                    org.apache.solr.client.solrj.SolrClient solrClient)
                             throws Exception
        Index content attachments as new entries in the idnex
        Parameters:
        collection - the collection of attachments
        content - the content whose attachments will be indexed
        solrClient - The solr client to use
        Throws:
        Exception - if something goes wrong when indexing the attachments of the content

      • indexContentAttachment

        public void indexContentAttachment​(Resource resource,
                                   Content content)
                            throws Exception
        Index a content attachment
        Parameters:
        resource - the content attachment as a Resource
        content - the content whose attachment is going to be indexed
        Throws:
        Exception - if something goes wrong when processing the indexation of the content attachment

      • indexContentAttachment

        public void indexContentAttachment​(Resource resource,
                                   Content content,
                                   org.apache.solr.client.solrj.SolrClient solrClient)
                            throws Exception
        Index a content attachment
        Parameters:
        resource - the content attachment as a Resource
        content - the content whose attachment is going to be indexed
        solrClient - The solr client to use
        Throws:
        Exception - if something goes wrong when processing the indexation of the content attachment

      • _indexResourceDocument

        protected void _indexResourceDocument​(Resource resource,
                                      org.apache.solr.common.SolrInputDocument document,
                                      org.apache.solr.client.solrj.SolrClient solrClient)
                               throws org.apache.solr.client.solrj.SolrServerException,
                                      IOException
        Index a populated solr input document of type Resource.
        Parameters:
        resource - the resource from which the input document is created
        document - the input document
        solrClient - The solr client to use
        Throws:
        org.apache.solr.client.solrj.SolrServerException - if there is an error on the server
        IOException - if there is a communication error with the server

      • deleteRepeaterDocs

        protected void deleteRepeaterDocs​(String contentId,
                                  String workspaceName,
                                  org.apache.solr.client.solrj.SolrClient solrClient)
                           throws Exception
        Delete repeater documents of a specified content.
        Parameters:
        workspaceName - The workspace name
        contentId - the content ID.
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • indexAllResources

        public Map<String,​Object> indexAllResources​(String workspaceName,
                                                  org.apache.solr.client.solrj.SolrClient solrClient)
                                           throws Exception
        Index all the resources in a given workspace.
        Parameters:
        workspaceName - The workspace where to index
        solrClient - The solr client to use
        Returns:
        The indexation result as a Map.
        Throws:
        Exception - if an error occurs while indexing.

      • indexResources

        public SolrIndexer.IndexationResult indexResources​(Iterable<AmetysObject> resources,
                                                   String documentType,
                                                   String workspaceName)
                                            throws Exception
        Send a collection of contents for indexation in the solr server.
        Parameters:
        resources - the collection of contents to index.
        documentType - The document type of the resource
        workspaceName - The workspace where to index
        Returns:
        the indexation result.
        Throws:
        Exception - if an error occurs while indexing.

      • indexResources

        public SolrIndexer.IndexationResult indexResources​(Iterable<? extends AmetysObject> resources,
                                                   String documentType,
                                                   TraversableAmetysObject resourceRoot,
                                                   String workspaceName,
                                                   org.apache.solr.client.solrj.SolrClient solrClient)
                                            throws Exception
        Send a collection of contents for indexation in the solr server.
        Parameters:
        resources - the collection of contents to index.
        documentType - The document type of the resource
        resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.
        workspaceName - The workspace where to index
        solrClient - The solr client to use
        Returns:
        the indexation result.
        Throws:
        Exception - if an error occurs while indexing.

      • doIndexResources

        protected SolrIndexer.IndexationResult doIndexResources​(Iterable<? extends AmetysObject> resources,
                                                        String documentType,
                                                        TraversableAmetysObject resourceRoot,
                                                        String workspaceName,
                                                        org.apache.solr.client.solrj.SolrClient solrClient)
                                                 throws Exception
        Send some resources for indexation in the solr server.
        Parameters:
        resources - the resources to index.
        documentType - The document type of the resource
        resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.
        workspaceName - The workspace where to index
        solrClient - The solr client to use
        Returns:
        the indexation result.
        Throws:
        Exception - if an error occurs committing the results.

      • indexResource

        public void indexResource​(Resource resource,
                          String documentType,
                          String workspaceName)
                   throws Exception
        Add or update a resource into Solr index
        Parameters:
        resource - The resource to index
        documentType - The document type of the resource
        workspaceName - The workspace where to index
        Throws:
        Exception - if an error occurs while indexing.

      • unindexResourcesByPath

        public void unindexResourcesByPath​(String rootId,
                                   String path)
                            throws Exception
        Delete all resource documents at a given path for all workspaces and commit
        Parameters:
        rootId - The resource root ID, must not be null.
        path - The resource path relative to the given root, must start with a slash.
        Throws:
        Exception - If an error occurs while unindexing.

      • unindexResourcesByPath

        public void unindexResourcesByPath​(String rootId,
                                   String path,
                                   String workspaceName)
                            throws Exception
        Delete all resource documents at a given path.
        Parameters:
        rootId - The resource root ID, must not be null.
        path - The resource path relative to the given root, must start with a slash.
        workspaceName - The workspace where to work in
        Throws:
        Exception - If an error occurs while unindexing.

      • unindexResource

        public void unindexResource​(String resourceId)
                     throws Exception
        Remove a resource from Solr index for all workspaces and commit
        Parameters:
        resourceId - The id of resource to unindex
        Throws:
        Exception - if an error occurs while indexing.

      • unindexResource

        public void unindexResource​(String resourceId,
                            String workspaceName)
                     throws Exception
        Remove a resource from the Solr index.
        Parameters:
        resourceId - The id of resource to unindex
        workspaceName - The workspace where to work in
        Throws:
        Exception - if an error occurs while unindexing.

      • doIndexExplorerItem

        protected void doIndexExplorerItem​(AmetysObject node,
                                   String documentType,
                                   TraversableAmetysObject resourceRoot,
                                   org.apache.solr.client.solrj.SolrClient solrClient)
                            throws Exception
        Add or update a resource into Solr index
        Parameters:
        node - The resource to index
        documentType - The document type of the resource
        resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • doIndexResource

        protected void doIndexResource​(Resource resource,
                               String documentType,
                               TraversableAmetysObject resourceRoot,
                               org.apache.solr.client.solrj.SolrClient solrClient)
                        throws Exception
        Add or update a resource into Solr index
        Parameters:
        resource - The resource to index
        documentType - The document type of the resource
        resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • commit

        public void commit()
            throws org.apache.solr.client.solrj.SolrServerException,
                   IOException
        Process a Solr commit operation in all workspaces.
        Use this only after a long operation with updates sent via NoAutoCommitUpdateClient
        Throws:
        org.apache.solr.client.solrj.SolrServerException - if there is an error on the server
        IOException - if there is a communication error with the server

      • commit

        public void commit​(String workspaceName,
                   org.apache.solr.client.solrj.SolrClient solrClient)
            throws org.apache.solr.client.solrj.SolrServerException,
                   IOException
        Process a Solr commit operation in given workspace.
        Parameters:
        workspaceName - The workspace's name
        solrClient - The solr client to use
        Throws:
        org.apache.solr.client.solrj.SolrServerException - if there is an error on the server
        IOException - if there is a communication error with the server

      • optimize

        public void optimize​(String workspaceName,
                     org.apache.solr.client.solrj.SolrClient solrClient)
              throws org.apache.solr.client.solrj.SolrServerException,
                     IOException
        Launch a solr index optimization.
        Parameters:
        workspaceName - The workspace's name
        solrClient - The solr client to use
        Throws:
        org.apache.solr.client.solrj.SolrServerException - if there is an error on the server
        IOException - if there is a communication error with the server

      • unindexAllDocuments

        public void unindexAllDocuments​(String workspaceName,
                                org.apache.solr.client.solrj.SolrClient solrClient)
                         throws Exception
        Delete all documents from the solr index.
        Parameters:
        workspaceName - The workspace name
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while unindexing.

      • doUnindexDocument

        protected void doUnindexDocument​(String id,
                                 String workspaceName,
                                 org.apache.solr.client.solrj.SolrClient solrClient)
                          throws Exception
        Delete a document from the Solr server.
        Parameters:
        id - The id of the document to delete from Solr
        workspaceName - The workspace name
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.

      • doUnindexContentAttachments

        protected void doUnindexContentAttachments​(String contentId,
                                           String workspaceName,
                                           org.apache.solr.client.solrj.SolrClient solrClient)
                                    throws Exception
        Delete content attachments documents of a given content from the Solr server.
        Parameters:
        contentId - The id of the content
        workspaceName - The workspace name
        solrClient - The solr client to use
        Throws:
        Exception - if an error occurs while indexing.