org.ametys.cms.content.indexing.solr

Class SolrIndexer

java.lang.Object

org.ametys.runtime.plugin.component.AbstractLogEnabled

org.ametys.cms.content.indexing.solr.SolrIndexer

All Implemented Interfaces:

LogEnabled, Initializable, Component, Contextualizable, Serviceable

public class SolrIndexer
extends AbstractLogEnabled
implements Component, Serviceable, Initializable, Contextualizable

Solr indexer.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) class	SolrIndexer.IndexationResult

Field Summary

Fields

Modifier and Type	Field and Description
private static ThreadLocal<SimpleDateFormat>	__DATE_FORMAT
private static String	_CONFIGSET_NAME
protected Context	_context The avalon context
private static List<String>	_READ_ONLY_FIELDS
private static List<String>	_READ_ONLY_FIELDTYPES
protected AmetysObjectResolver	_resolver The ametys object resolver.
protected SchemaDefinitionProviderExtensionPoint	_schemaDefProviderEP The schema definition provider extension point.
protected SchemaHelper	_schemaHelper The schema helper.
protected SolrClientProvider	_solrClientProvider The Solr client provider
protected SolrContentIndexer	_solrContentIndexer Solr Ametys contents indexer
protected SolrContentRightIndexer	_solrContentRightIndexer Solr right indexer.
protected String	_solrCorePrefix The solr core prefix.
protected SolrResourceIndexer	_solrResourceIndexer Solr resource indexer.
protected SolrWorkflowIndexer	_solrWorkflowIndexer Solr workflow indexer.
protected WorkspaceSelector	_workspaceSelector The workspace selector.
static String	ROLE The component role.

Constructor Summary

Constructors

Constructor and Description
SolrIndexer()

Method Summary

Modifier and Type	Method and Description
private SolrClient	_defaultSolrClient()
private void	_doIndexContent(Content content, String workspaceName, boolean commit)
protected SolrClient	_getSolrClient(String workspaceName) Gets the Solr client
protected void	addCustomUpdates(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates, SchemaFields schemaFields) Compute the list of custom SchemaRequest.Update directives.
protected void	addStaticSchemaUpdates(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates, org.apache.solr.client.solrj.response.schema.SchemaRepresentation staticSchema, SchemaFields schemaFields) Compute the list of SchemaRequest.Update directives from the static schema.
protected void	clearSchema(SolrClient solrClient, String collection) Delete all the fields of the existing schema in the given collection.
void	commit(String workspaceName) Process a Solr commit operation in given workspace.
void	contextualize(Context context)
void	createCore(String name) Create a Solr core.
static SimpleDateFormat	dateFormat() Returns the formatter for indexing dates.
protected boolean	definitionExists(SchemaDefinition definition, SchemaFields schemaFields) Test if the given schema definition exists in the given SchemaFields reference.
void	deleteCore(String name) Delete a Solr core.
protected void	deleteRepeaterDocs(String contentId, String workspaceName) Delete repeater documents of a specified content.
protected void	doIndexContent(Content content, String workspaceName) Add or update a content into Solr index
protected void	doIndexContentAcls(Content content, SolrInputDocument document) Index the users and groups who are allowed to access the content.
protected SolrIndexer.IndexationResult	doIndexContents(Iterable<Content> contents, String workspaceName, boolean commit) Send some contents for indexation in the solr server.
protected void	doIndexContentWorkflow(Content content, String workspaceName, boolean commit) Index the whole workflow of a content.
protected void	doIndexExplorerItem(AmetysObject node, String documentType, TraversableAmetysObject resourceRoot) Add or update a resource into Solr index
protected void	doIndexResource(Resource resource, String documentType, TraversableAmetysObject resourceRoot) Add or update a resource into Solr index
protected SolrIndexer.IndexationResult	doIndexResources(Iterable<? extends AmetysObject> resources, String documentType, TraversableAmetysObject resourceRoot, String workspaceName, boolean commit) Send some resources for indexation in the solr server.
protected void	doUnindexDocument(String id, String workspaceName) Delete a document from the Solr server.
Set<String>	getCoreNames() Get the names of the Solr cores.
protected Set<String>	getRealCoreNames() Get the names of the Solr cores.
protected org.apache.solr.client.solrj.request.schema.SchemaRequest.Update	getSchemaUpdate(SchemaDefinition definition) Get the schema SchemaRequest.Update directive from the given schema definition.
Map<String,Object>	indexAllContents(String workspaceName, boolean commit) Index all the contents in a given workspace.
Map<String,Object>	indexAllResources(String workspaceName, boolean commit) Index all the resources in a given workspace.
void	indexContent(String contentId) Add or update a content into Solr index on all workspaces and commit
void	indexContent(String contentId, String workspaceName, boolean commit) Add or update a content into Solr index
SolrIndexer.IndexationResult	indexContents(Iterable<Content> contents) Send a collection of contents for indexation in the solr server on all workspaces and commit
SolrIndexer.IndexationResult	indexContents(Iterable<Content> contents, String workspaceName, boolean commit) Send a collection of contents for indexation in the solr server.
void	indexResource(Resource resource, String documentType, String workspaceName, boolean commit) Add or update a resource into Solr index
SolrIndexer.IndexationResult	indexResources(Iterable<? extends AmetysObject> resources, String documentType, TraversableAmetysObject resourceRoot, String workspaceName, boolean commit) Send a collection of contents for indexation in the solr server.
SolrIndexer.IndexationResult	indexResources(Iterable<AmetysObject> resources, String documentType, String workspaceName, boolean commit) Send a collection of contents for indexation in the solr server.
void	indexSubcontents(String collectionId) Add or update the child contents of a AmetysObjectCollection into Solr index, for all workspaces and commit
void	indexSubcontents(String collectionId, String workspaceName, boolean commit) Index the child contents of a AmetysObjectCollection
void	initialize()
void	optimize(String workspaceName) Launch a solr index optimization.
protected void	reloadCores() Reload the solr cores.
void	rollback(String workspaceName) Process a Solr rollback operation.
void	sendSchema() Send the schema.
void	service(ServiceManager serviceManager)
protected void	unindexAllContents(String workspaceName, boolean commit) Unindex all content documents.
void	unindexAllDocuments(String workspaceName, boolean commit) Delete all documents from the solr index.
void	unindexAllResources(String workspaceName, boolean commit) Delete all resource documents from the solr index.
void	unindexContent(String contentId) Remove a content from Solr index for all workspaces and commit
void	unindexContent(String contentId, String workspaceName, boolean commit) Remove a content from Solr index
void	unindexContents(Collection<String> contentIds) Remove a content from Solr index for all workspaces and commit
void	unindexContents(Collection<String> contentIds, String workspaceName, boolean commit) Remove a content from Solr index
void	unindexResource(String resourceId) Remove a resource from Solr index for all workspaces and commit
void	unindexResource(String resourceId, String workspaceName, boolean commit) Remove a resource from the Solr index.
void	unindexResourcesByPath(String rootId, String path) Delete all resource documents at a given path for all workspaces and commit
void	unindexResourcesByPath(String rootId, String path, String workspaceName, boolean commit) Delete all resource documents at a given path.
void	updateSystemProperty(Content content, String propertyId, String workspaceName, boolean commit) Update the value of a specific system property in a content document.

Methods inherited from class org.ametys.runtime.plugin.component.AbstractLogEnabled

getLogger, setLogger

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ROLE

public static final String ROLE

The component role.

__DATE_FORMAT

private static final ThreadLocal<SimpleDateFormat> __DATE_FORMAT

_CONFIGSET_NAME

private static final String _CONFIGSET_NAME

See Also:

Constant Field Values

_READ_ONLY_FIELDS

private static final List<String> _READ_ONLY_FIELDS

_READ_ONLY_FIELDTYPES

private static final List<String> _READ_ONLY_FIELDTYPES

_resolver

protected AmetysObjectResolver _resolver

The ametys object resolver.

_schemaDefProviderEP

protected SchemaDefinitionProviderExtensionPoint _schemaDefProviderEP

The schema definition provider extension point.

_schemaHelper

protected SchemaHelper _schemaHelper

The schema helper.

_solrContentIndexer

protected SolrContentIndexer _solrContentIndexer

Solr Ametys contents indexer

_solrWorkflowIndexer

protected SolrWorkflowIndexer _solrWorkflowIndexer

Solr workflow indexer.

_solrContentRightIndexer

protected SolrContentRightIndexer _solrContentRightIndexer

Solr right indexer.

_solrResourceIndexer

protected SolrResourceIndexer _solrResourceIndexer

Solr resource indexer.

_solrClientProvider

protected SolrClientProvider _solrClientProvider

The Solr client provider

_solrCorePrefix

protected String _solrCorePrefix

The solr core prefix.

_workspaceSelector

protected WorkspaceSelector _workspaceSelector

The workspace selector.

_context

protected Context _context

The avalon context

Constructor Detail

SolrIndexer

public SolrIndexer()

Method Detail

dateFormat

public static SimpleDateFormat dateFormat()

Returns the formatter for indexing dates. This is used for adding a dates as formatted strings (and not with date object directly) to prevent indexing of the wrong value because of time zone

Returns:

The date format for indexing dates

service

public void service(ServiceManager serviceManager)
             throws ServiceException

Specified by:

service in interface Serviceable

Throws:

ServiceException

initialize

public void initialize()
                throws Exception

Specified by:

initialize in interface Initializable

Throws:

Exception

contextualize

public void contextualize(Context context)
                   throws ContextException

Specified by:

contextualize in interface Contextualizable

Throws:

ContextException

_getSolrClient

protected SolrClient _getSolrClient(String workspaceName)

Gets the Solr client

Parameters:

workspaceName - The name of the workspace

Returns:

the Solr client

_defaultSolrClient

private SolrClient _defaultSolrClient()

getCoreNames

public Set<String> getCoreNames()
                         throws IOException,
                                SolrServerException

Get the names of the Solr cores.

Returns:

The names of the Solr cores.

Throws:

IOException - If an I/O error occurs.

SolrServerException - If a Solr error occurs.

getRealCoreNames

protected Set<String> getRealCoreNames()
                                throws IOException,
                                       SolrServerException

Get the names of the Solr cores.

Returns:

The names of the Solr cores.

Throws:

IOException - If an I/O error occurs.

SolrServerException - If a Solr error occurs.

createCore

public void createCore(String name)
                throws IOException,
                       SolrServerException

Create a Solr core.

Parameters:

name - The name of the core to create.

Throws:

IOException - If an I/O error occurs.

SolrServerException - If a Solr error occurs.

deleteCore

public void deleteCore(String name)
                throws IOException,
                       SolrServerException

Delete a Solr core.

Parameters:

name - The name of the core to delete.

Throws:

IOException - If an I/O error occurs.

SolrServerException - If a Solr error occurs.

sendSchema

public void sendSchema()
                throws IOException,
                       SolrServerException

Send the schema.

Throws:

IOException - If a communication error occurs.

SolrServerException - If a solr error occurs.

addStaticSchemaUpdates

protected void addStaticSchemaUpdates(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates,
                                      org.apache.solr.client.solrj.response.schema.SchemaRepresentation staticSchema,
                                      SchemaFields schemaFields)

Compute the list of SchemaRequest.Update directives from the static schema.

Parameters:

updates - The list of SchemaRequest.Update directives to fill.

staticSchema - The static schema representation.

schemaFields - The current schema fields, used to track the existing fields (to be filled).

addCustomUpdates

protected void addCustomUpdates(List<org.apache.solr.client.solrj.request.schema.SchemaRequest.Update> updates,
                                SchemaFields schemaFields)

Compute the list of custom SchemaRequest.Update directives.

Parameters:

updates - The list of SchemaRequest.Update directives to fill.

schemaFields - The current schema fields, used to track the existing fields (to be filled).

definitionExists

protected boolean definitionExists(SchemaDefinition definition,
                                   SchemaFields schemaFields)

Test if the given schema definition exists in the given SchemaFields reference.

Parameters:

definition - The schema definition to test.

schemaFields - The current schema fields.

Returns:

true if the SchemaFields contain the schema definition.

clearSchema

protected void clearSchema(SolrClient solrClient,
                           String collection)
                    throws IOException,
                           SolrServerException

Delete all the fields of the existing schema in the given collection.

Parameters:

solrClient - The Solr client

collection - The collection.

Throws:

IOException - If a communication error occurs.

SolrServerException - If a solr error occurs.

getSchemaUpdate

protected org.apache.solr.client.solrj.request.schema.SchemaRequest.Update getSchemaUpdate(SchemaDefinition definition)

Get the schema SchemaRequest.Update directive from the given schema definition.

Parameters:

definition - The schema definition to add.

Returns:

The add SchemaRequest.Update directive.

reloadCores

protected void reloadCores()
                    throws IOException,
                           SolrServerException

Reload the solr cores.

Throws:

IOException - If a communication error occurs.

SolrServerException - If a solr error occurs.

indexAllContents

public Map<String,Object> indexAllContents(String workspaceName,
                                           boolean commit)
                                    throws Exception

Index all the contents in a given workspace.

Parameters:

workspaceName - the workspace where to index

commit - true to commit

Returns:

The indexation result as a Map.

Throws:

Exception - if an error occurs while indexing.

unindexAllContents

protected void unindexAllContents(String workspaceName,
                                  boolean commit)
                           throws Exception

Unindex all content documents.

Parameters:

workspaceName - The workspace name

commit - true to commit

Throws:

Exception - if an error occurs while unindexing.

indexSubcontents

public void indexSubcontents(String collectionId)
                      throws Exception

Add or update the child contents of a AmetysObjectCollection into Solr index, for all workspaces and commit

Parameters:

collectionId - The id of collection

Throws:

Exception - if an error occurs while indexing.

indexSubcontents

public void indexSubcontents(String collectionId,
                             String workspaceName,
                             boolean commit)
                      throws Exception

Index the child contents of a AmetysObjectCollection

Parameters:

collectionId - The id of collection

workspaceName - the workspace where to index

commit - true to commit

Throws:

Exception - if an error occurs while unindexing.

indexContent

public void indexContent(String contentId)
                  throws Exception

Add or update a content into Solr index on all workspaces and commit

Parameters:

contentId - The id of the content to index

Throws:

Exception - if an error occurs while indexing.

indexContent

public void indexContent(String contentId,
                         String workspaceName,
                         boolean commit)
                  throws Exception

Add or update a content into Solr index

Parameters:

contentId - The id of the content to index

workspaceName - the workspace where to index

commit - true to commit the indexation

Throws:

Exception - if an error occurs while indexing.

_doIndexContent

private void _doIndexContent(Content content,
                             String workspaceName,
                             boolean commit)
                      throws IndexingException

Throws:

IndexingException

indexContents

public SolrIndexer.IndexationResult indexContents(Iterable<Content> contents)
                                           throws Exception

Send a collection of contents for indexation in the solr server on all workspaces and commit

Parameters:

contents - the collection of contents to index.

Returns:

the indexation result.

Throws:

Exception - if an error occurs while indexing.

indexContents

public SolrIndexer.IndexationResult indexContents(Iterable<Content> contents,
                                                  String workspaceName,
                                                  boolean commit)
                                           throws Exception

Send a collection of contents for indexation in the solr server.

Parameters:

contents - the collection of contents to index.

workspaceName - the workspace where to index

commit - true to commit the indexation

Returns:

the indexation result.

Throws:

Exception - if an error occurs while indexing.

doIndexContents

protected SolrIndexer.IndexationResult doIndexContents(Iterable<Content> contents,
                                                       String workspaceName,
                                                       boolean commit)
                                                throws Exception

Send some contents for indexation in the solr server.

Parameters:

contents - the contents to index.

workspaceName - The workspace name

commit - true to commit transaction

Returns:

the indexation result.

Throws:

Exception - if an error occurs committing the results.

updateSystemProperty

public void updateSystemProperty(Content content,
                                 String propertyId,
                                 String workspaceName,
                                 boolean commit)
                          throws Exception

Update the value of a specific system property in a content document.

Parameters:

content - The content to update.

propertyId - The system property ID.

workspaceName - The workspace name

commit - true to commit update

Throws:

Exception - if an error occurs while indexing.

unindexContent

public void unindexContent(String contentId)
                    throws Exception

Remove a content from Solr index for all workspaces and commit

Parameters:

contentId - The id of content to unindex

Throws:

Exception - if an error occurs while indexing.

unindexContent

public void unindexContent(String contentId,
                           String workspaceName,
                           boolean commit)
                    throws Exception

Remove a content from Solr index

Parameters:

contentId - The id of content to unindex

workspaceName - The workspace where to work in

commit - true to commit operation

Throws:

Exception - if an error occurs while indexing.

unindexContents

public void unindexContents(Collection<String> contentIds)
                     throws Exception

Remove a content from Solr index for all workspaces and commit

Parameters:

contentIds - The id of content to unindex

Throws:

Exception - if an error occurs while indexing.

unindexContents

public void unindexContents(Collection<String> contentIds,
                            String workspaceName,
                            boolean commit)
                     throws Exception

Remove a content from Solr index

Parameters:

contentIds - The id of content to unindex

workspaceName - The workspace where to work in

commit - true to commit

Throws:

Exception - if an error occurs while indexing.

doIndexContent

protected void doIndexContent(Content content,
                              String workspaceName)
                       throws Exception

Add or update a content into Solr index

Parameters:

content - The content to index

workspaceName - The workspace where to index

Throws:

Exception - if an error occurs while indexing.

doIndexContentAcls

protected void doIndexContentAcls(Content content,
                                  SolrInputDocument document)
                           throws Exception

Index the users and groups who are allowed to access the content.

Parameters:

content - The content.

document - The document.

Throws:

Exception - If an error occurs while indexing.

doIndexContentWorkflow

protected void doIndexContentWorkflow(Content content,
                                      String workspaceName,
                                      boolean commit)
                               throws Exception

Index the whole workflow of a content.

Parameters:

content - The content.

workspaceName - The workspace name

commit - true to commit, false otherwise.

Throws:

Exception - if an error occurs while indexing.

deleteRepeaterDocs

protected void deleteRepeaterDocs(String contentId,
                                  String workspaceName)
                           throws Exception

Delete repeater documents of a specified content.

Parameters:

workspaceName - The workspace name

contentId - the content ID.

Throws:

Exception - if an error occurs while indexing.

indexAllResources

public Map<String,Object> indexAllResources(String workspaceName,
                                            boolean commit)
                                     throws Exception

Index all the resources in a given workspace.

Parameters:

workspaceName - The workspace where to index

commit - true to commit indexation

Returns:

The indexation result as a Map.

Throws:

Exception - if an error occurs while indexing.

indexResources

public SolrIndexer.IndexationResult indexResources(Iterable<AmetysObject> resources,
                                                   String documentType,
                                                   String workspaceName,
                                                   boolean commit)
                                            throws Exception

Send a collection of contents for indexation in the solr server.

Parameters:

resources - the collection of contents to index.

documentType - The document type of the resource

workspaceName - The workspace where to index

commit - true to commit indexation

Returns:

the indexation result.

Throws:

Exception - if an error occurs while indexing.

indexResources

public SolrIndexer.IndexationResult indexResources(Iterable<? extends AmetysObject> resources,
                                                   String documentType,
                                                   TraversableAmetysObject resourceRoot,
                                                   String workspaceName,
                                                   boolean commit)
                                            throws Exception

Send a collection of contents for indexation in the solr server.

Parameters:

resources - the collection of contents to index.

documentType - The document type of the resource

resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.

workspaceName - The workspace where to index

commit - true to commit indexation

Returns:

the indexation result.

Throws:

Exception - if an error occurs while indexing.

doIndexResources

protected SolrIndexer.IndexationResult doIndexResources(Iterable<? extends AmetysObject> resources,
                                                        String documentType,
                                                        TraversableAmetysObject resourceRoot,
                                                        String workspaceName,
                                                        boolean commit)
                                                 throws Exception

Send some resources for indexation in the solr server.

Parameters:

resources - the resources to index.

documentType - The document type of the resource

resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.

workspaceName - The workspace where to index

commit - true to commit indexation

Returns:

the indexation result.

Throws:

Exception - if an error occurs committing the results.

indexResource

public void indexResource(Resource resource,
                          String documentType,
                          String workspaceName,
                          boolean commit)
                   throws Exception

Add or update a resource into Solr index

Parameters:

resource - The resource to index

documentType - The document type of the resource

workspaceName - The workspace where to index

commit - true to commit indexation

Throws:

Exception - if an error occurs while indexing.

unindexAllResources

public void unindexAllResources(String workspaceName,
                                boolean commit)
                         throws Exception

Delete all resource documents from the solr index.

Parameters:

workspaceName - The workspace name

commit - true to commit deletion

Throws:

Exception - If an error occurs while unindexing.

unindexResourcesByPath

public void unindexResourcesByPath(String rootId,
                                   String path)
                            throws Exception

Delete all resource documents at a given path for all workspaces and commit

Parameters:

rootId - The resource root ID, must not be null.

path - The resource path relative to the given root, must start with a slash.

Throws:

Exception - If an error occurs while unindexing.

unindexResourcesByPath

public void unindexResourcesByPath(String rootId,
                                   String path,
                                   String workspaceName,
                                   boolean commit)
                            throws Exception

Delete all resource documents at a given path.

Parameters:

rootId - The resource root ID, must not be null.

path - The resource path relative to the given root, must start with a slash.

workspaceName - The workspace where to work in

commit - true to commit operation

Throws:

Exception - If an error occurs while unindexing.

unindexResource

public void unindexResource(String resourceId)
                     throws Exception

Remove a resource from Solr index for all workspaces and commit

Parameters:

resourceId - The id of resource to unindex

Throws:

Exception - if an error occurs while indexing.

unindexResource

public void unindexResource(String resourceId,
                            String workspaceName,
                            boolean commit)
                     throws Exception

Remove a resource from the Solr index.

Parameters:

resourceId - The id of resource to unindex

workspaceName - The workspace where to work in

commit - true to commit operation

Throws:

Exception - if an error occurs while unindexing.

doIndexExplorerItem

protected void doIndexExplorerItem(AmetysObject node,
                                   String documentType,
                                   TraversableAmetysObject resourceRoot)
                            throws Exception

Add or update a resource into Solr index

Parameters:

node - The resource to index

documentType - The document type of the resource

resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.

Throws:

Exception - if an error occurs while indexing.

doIndexResource

protected void doIndexResource(Resource resource,
                               String documentType,
                               TraversableAmetysObject resourceRoot)
                        throws Exception

Add or update a resource into Solr index

Parameters:

resource - The resource to index

documentType - The document type of the resource

resourceRoot - The resource root, can be null. In that case, it will be computed for each resource.

Throws:

Exception - if an error occurs while indexing.

commit

public void commit(String workspaceName)
            throws SolrServerException,
                   IOException

Process a Solr commit operation in given workspace.

Parameters:

workspaceName - The workspace's name

Throws:

SolrServerException - if there is an error on the server

IOException - if there is a communication error with the server

rollback

public void rollback(String workspaceName)
              throws SolrServerException,
                     IOException

Process a Solr rollback operation.

Parameters:

workspaceName - The workspace's name

Throws:

SolrServerException - if there is an error on the server

IOException - if there is a communication error with the server

optimize

public void optimize(String workspaceName)
              throws SolrServerException,
                     IOException

Launch a solr index optimization.

Parameters:

workspaceName - The workspace's name

Throws:

SolrServerException - if there is an error on the server

IOException - if there is a communication error with the server

unindexAllDocuments

public void unindexAllDocuments(String workspaceName,
                                boolean commit)
                         throws Exception

Delete all documents from the solr index.

Parameters:

workspaceName - The workspace name

commit - true to commit

Throws:

Exception - if an error occurs while unindexing.

doUnindexDocument

protected void doUnindexDocument(String id,
                                 String workspaceName)
                          throws Exception

Delete a document from the Solr server.

Parameters:

id - The id of the document to delete from Solr

workspaceName - The workspace name

Throws:

Exception - if an error occurs while indexing.