Software testing is a process that involves making sure the client’s requirements are satisfied and that a fully functional solution is delivered. To ensure that your testing validates the solution is working correctly and meets specification, it is vital that you determine what functionality you want to test, and how the solution should function before you begin.
This is done through the use of test cases (or scenarios), so to get you started I would like to share with you some tips on what to think about when testing, using the example of a search tool.
Note: This is a blog originating back from 2012. If you came here because you are looking for better search experiences in SharePoint Online or Microsoft 365, then you may want to check out the related blogs at the bottom of this page, or these other pages linked here:
Although there are differences between software platforms and different types of searches, fundamentally search works by an automated function (typically referred to as either a crawler, robot or indexer) “crawling” over the content in a system and indexing the results.
When the robots find a piece of content they add details about it into the “Index”, which is kind of like a short hand summary of all the systems content. When you perform a search, the search engine then uses one of those famous algorithms that you hear about (For example: People say that Google’s Algorithm “Rules the Web”) to scan through the Index, and find the appropriate results. The results shown from the index then link to the real content.
So knowing this, we need to identify what can be tested. The test cases depend on the documentation provided by the client and their requirements. The more situations you can think of, and the more tests you can run within your time frame, add more value to your test plan.
Test Case / Scenario Examples
Search Values:
- Find out the minimum and maximum range of variables in the search query (how long can my test string be?)
- Find out the depth of the search – does it search document titles? For words inside document? How does image search work?
- Are there advanced search features like properties? Where we can search by document or image type?
- Does search does work if no keywords are entered?
- Test that search works for capital letters (is it case sensitive? Should it be?)
- Does search work for numbers?
- Should search work for special characters? If I just search for a symbol, do I get results? Are results impacted by using special characters (such as XOR approach)
- Do you have wildcard searching? Enter “a*” and ensure that the results display all characters starting with ‘a’. Try pressing “Enter” key instead of clicking “Search button”
Search Results (do the results reflect your search parameters?)
- Do you have phonetic searching? So if a keyword is typed incorrectly, then the relevant result message should be displayed. For example, if ?search !Func^tionality} is entered into the search box, the search ‘functionality’ results should be displayed.
- How does navigating back and forward in a search impact the results? Does the page expire? Should it?
- Search response time should be checked, does this meet performance expectations?
- Do the search results look as expected? The specification states each search result should contain a link and a few lines containing the searched keywords and the link should navigate to the page where the keywords exist, do they look like that?
- Depending on the search tool, searched keyword should be highlighted in the search result page and also in the page where the keyword exists
- Check the total number of results that are displayed on the search result page, does this look correct?
- Do you have results refiners? Are the correct results refiners showing on the screen, when you use a search refiner, and the results refined correctly?
Advanced Tests
- Search Security: Some content in the system may be hidden from users, but when they search can they see these items? Change the security settings on some content, see if it disappears from the search results. In most internal systems, such as SharePoint, you should expect search results to be security trimmed.
- Search Indexing: The crawler must find and index new content, how long is this supposed to take? Add a new item, see how long it takes before you find it in the search results.
Depending on your clients’ specification, this list can be even longer!