| If you read my blog, follow me on Twitter, or sometimes meet me at conferences, SharePoint events, you might know I’ve had a lot of changes in my life in the past 1.5 years. But it’s now public: there is one more, big change you didn’t know so far. This story begins back in the Summer of 2010, when I’ve started to work as an Independent Consultant after leaving the company I was a co-owner and co-founder of. Short afterwards, I joined BA Insight as a part-time subcontractor. It’s been a pleasure to become the part of this great team: I’ve been focusing on Enterprise Search and Content Management for many-many years and here I could really get improved, learn a lot and contribute to exciting projects and products. It’s amazing when you can learn and experience something new every day! When you can contribute to amazing products. When you can work with great people. As months passed away, I had to realize I love to work here more and more, day by day. I’ve been involved not only to more and more cool projects, but also to the work of product teams “behind the scenes”. And finally it turned out it was like an engagement – I’ve been spending more and more time with BA Insight and wasn’t even thinking about my “independent consultant” life. And then, in this January, the “marriage” happened: I joined the company as a full time Senior Solutions Consultant and Project Leader and I have to say: I found what I was working for in the past more than 10 years. “In her role, Molnar will help BA Insight’s enterprise customers craft and deploy search solutions that enhance productivity and reduce costs. Additionally, Molnar will play a pivotal role in BA Insight’s efforts to develop innovative technologies that help organizations leverage the full power of Microsoft SharePoint and FAST Search across the enterprise.” (by BA Insight News and Press Releases) |
| I’ve got a lot of questions about my life, my past and my personality recently, so that I’ve decided to share some stories with you. Not because I’d like to show you how ‘good’ I am but to show you: it’s possible to change your life if you really want it. So, let me start with the years at the university. I’ve made my high school in a small Hungarian town’s small high school, with special classes on Math and Physics. My parents were very poor and the school was very small and not too good, but I had some VERY good teachers there who I will be grateful in my whole life. They were supporting me in my ‘crazy’ computer mania, they let me in to the computer room whenever I wanted, they gave me a lot of books (don’t forget: by that time we hadn’t have Internet yet), etc. I was the very first student in the school who had email address but couldn’t use that as hadn’t have anybody with email address around me yet. I made a lot of cool programs there, including a German-Hungarian dictionary with more than 26,000 words, some e-learning programs for the subjects I didn’t like (history, for example), chat program over a Norton server with one of my friends (who was a computer fan too, of course), etc.
I’ve had a lot of fun there. Then I had to decide what’s next. For me, it was obvious to choose the best university in the country, so that applied to the Budapest University of Technology and Economics (BUTE) and I’ve got accepted immediately. It was one of the biggest wonder in my life, I could leave the small town and moved to Budapest! Wow… But after a few months, my parents realized they couldn’t afford my education. After the biggest wonder the darkest days came.
Then I made the decision to start working instead of leaving the school. I didn’t left my class but started to work as a developer at Graphisoft in parallel.
I have had mixed feelings about this.
First, it was amazing to work for such a great company. At the beginning I had to work with GDL, ArchiCad’s Graphical Description Language – can you imagine what a fun to write a code that finally draws the bridges of Budapest? My favorite one is still the Elisabeth Bridge because of its simply math model and beauty. Then, I had to improve GDL. You know, programming the framework for a program language. Sounds good, right? Yes, it was an amazing job, amazing time and I could learn a lot. But at the same time, you know, I still was a student, had to learn (sometimes should have learn much more…), and yes, as every typical student I wanted to socialize much more. While my friends went out, I was working to be able to stay at the university. While definitely learned a lot, like time management, self-discipline, working hard, etc., it was a hard, very hard time of my life. When .NET came out, I’ve been working for a T-System company as a developer and got the task to ‘explore’ the new language and framework. Then, I’ve got a kind of new ‘research’ task: me and two of my colleagues had to make a .NET forms application that was able to make operations on Exchange public folders. Operations, like Search. With the API of something that was called “Tahoe”… It was my very first meeting with SharePoint. I didn’t know what was this, I didn’t even know it had any UI, the only thing I new there was an API that had Search functions and I had to drive it to be able to work over my Exchange public folders… A few months later, my managers decided to give me a new project. You know, like “you already know SharePoint”… Then one more, one more, one more SharePoint project even after some company changes again, but I always have got bigger and bigger tasks, projects so I had to explore the product more and more. But still, everybody though I knew SharePoint even before I knew about that… By the time I finished my studies as M.Sc. of Computer Sciences I was working as a software architect and had my own SharePoint classes in a training center. I still didn’t know I wanted to focus on SharePoint, but SharePoint wanted me way too much… |
| Refiners are cool either if you use SharePoint Search or FAST, it’s not a question. I very like them, they give so much options and power to the end users. But there’s a very common question around them: the deep and shallow behavior. As you know the definitions very well: FAST Search for SharePoint has deep refiners, that means each result in the result set is processed and used when calculating the refiners. And SharePoint Search uses shallow refiners, where the refiner values are calculated from the first 50 results only. These definitions are easy, right? But let’s think a bit forward, and try to answer the question that pops up at almost every conference: Why some Refiner values are not visible when searching? Moreover: why they’re visible when running Query1 and hidden when running Query2? For example: let’s say you have a lot of documents crawled, and you enter a query where the result set contains many-many items. Thousands, tens of thousands or even more. Let’s say you have some Excel workbook in the result set that might be relevant for you, but this Excel file is not boosted in the result set at all, let’s say the first Excel result is on the 51th position (you have a lot of Word, PowerPoint, PDF, etc. files on the positions 1-50). What happens if you use FAST Search? – As the refiners are deep, each result will be processed, so your Excel workbook. For example, in the Result Type refiner you’ll see all the Word, PowerPoint, PDF file types as well as the Excel. Easy way, you can click on the Excel refiner and you’ll get what you’re looking for immediately. 
But what’s the case if you don’t have FAST Search, only the SharePoint one? – As the first 50 results is processed for the refiner calculation, your Excel workbook won’t be included. This means, the Result Type refiner displays the Word, PowerPoint, PDF refiners but doesn’t display the Excel at all, as your Excel file is not amongst the top results. You’ll see the Result Type refiner as if it there wasn’t any Excel result at all! 
Conclusion: the difference between the shallow and deep refiners doesn’t seem to be so much important for the first sight. But you have to be aware there’s a huge difference in a real production environment as you and your users might have some “hidden” refiners, and sometimes it’s hard to understand why. In other words, if a refiner value shows up on your Refinement Panel, that means: - In case of FAST Search for SharePoint (deep refiner): There’s at least one item matching this refiner value in the whole result set. Exact number of the items match the refiner value is included.
- In case of SharePoint Search (shallow refiner): There’s at least one item matching this refiner value in the first 50 results.
If you cannot see a specific value on the Refiner Panel, that means: - In case of FAST Search for SharePoint (deep refiner): There’s no result matching this refiner value at all.
- In case of SharePoint Search (shallow refiner): There’s no result matching this refiner in the first 50 results.
|
| Recently, I’ve made a webinar with Dave Coleman and MetaVis. Thanks to the great people attending, I’ve got lot more question I was able to answer then, so as an ultimate solution I promised to answer them on my blog. So here are my answers, part #1: - What is the motivation behind prevent folders under Document Sets - is it search related or just an business level decision? – Unfortunately I don’t know the exact answer. It was a technical decision during design time, as I know. The property management and all inheritance things could be much more complicated in case we could have a hierarchy inside a Document Set.
- Will you provide the ppt's after the webinar? – Yes, the PPTs have been uploaded to slideshare: http://www.slideshare.net/aghy/information-architecture-searchbettertogether-8814725
- Can the content uploaded and processed by content organizer be moved to different libraries in different site collections? – Out of the box, the Content Organizer Rules can move the documents inside a Site Collection. If you need to move them out, you need some custom solution.
- Is SharePoint 2010 search based on FAST? – No, FAST Search for SharePoint is based on the FAST Search product line, and SP2010 Search is based on the previous SharePoint products’ code.
- Can we import Term sets from external system like external data in other software running in the company like we can do this in via bcs? – Yes, you can import term sets in CSV format. You can find more information here: http://technet.microsoft.com/en-us/library/ee424396.aspx
- Have we been seeing any BA-Insight Longitude Search additions to FAST Search or was it all only FASt Search? – No, all my demos contained out-of-the-box capabilities only (both SP2010 and FAST Search).
- Is Search also limited to a single Site (Server) or could this be run over multiple Sites (Servers) at same time? – You can scale-out your Crawler and Query servers to multiple servers both in case of SP2010 Search and FAST Search.
- How do activate FAST Search? – FAST Search Server 2010 for SharePoint (FS4SP) is a separated product. Steps for “activating” FS4SP are the followings:
- You must have a SharePoint 2010 Enterprise farm.
- Install FAST Search server(s). Depending on your environment, you can have one or more FS4SP servers, logically it’s a farm too.
- Configure your SP2010 farm and the FS4SP farm to work together, create the Search Server Applications in SharePoint.
- If you’re interested in much more details, here are some useful links and troubleshooting steps for you: http://aghy.hu/AghyBlog_EN/Lists/Posts/Post.aspx?ID=119
- How to better integrate search in the custom applications (not from inside sharepoint)? – SharePoint has both an API (Object Model) and a collection of Web Services. You can call those Web Services from any remote application (with proper permissions, of course), while Object Model can be used on the SharePoint server only. You can find the QueryService reference here: http://msdn.microsoft.com/en-us/library/websvcsearch.queryservice_members.aspx
More to come soon (sonner than this one, I can tell you)… |
| Recently, I have seen several “issues” after installing SharePoint 2010 SP1. The fix is very easy, but first let me describe the symptoms I have seen. 1. “The search application ‘Search Service Application’ on server MYSERVER did not finish loading. View the event logs on the affected server for more information.” This error appears on the Search Service Application. Event Log on the server contains tons of errors like this: Log Name: Application Source: Microsoft-SharePoint Products-SharePoint Server Date: 7/19/2011 5:01:22 PM Event ID: 6481 Task Category: Shared Services Level: Error Keywords: User: MYDOMAIN\svcuser Computer: myserver.mydomain.local Description: Application Server job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (898667e4-126e-45d2-bb52-43f613669084). Reason: The device is not ready. Technical Support Details: System.IO.FileNotFoundException: The device is not ready. at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize() at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob) 2. On a different server, the SharePoint needed to be upgraded from Standard to Enterprise (after SP1 has just been installed). After entering the license key, we got an “Unsuccessful” error on the User Interface, and this in the Event Log: The synchronization operation of the search component: d289abde-9641-46d0-8d32-0345f1885704 associated to the search application: Search Service Application on server: BAIQATEST01 has failed. The component version is not compatible with the search database: Search_Service_Application_CrawlStoreDB_a737e7614f034544a8c1da6fe4a24f7b on server: BAIQATEST01. The possible cause for this failure is The database schema version is less than the minimum backwards compatibility schema version that is supported for this component. To resolve this problem upgrade this database. The reason of these errors were the same in both cases: SharePoint 2010 Config Wizard should have been run after installing SP1. Once you run the Config Wizard, these errors disappear and everything starts to work fine – again! |
| As you know I’m playing working with SharePoint/FAST Search a lot. I have a lot of tasks when I have to sit on the button F5 while crawling and check the status: is it started? is it still crawling? is it finished yet?… I have to hit F5 in every minute. I’m too lazy, so decided to write a PowerShell script that does nothing but checking the crawl status of a Content Source and writes it to the console to me. And I can work on my second screen while it’s working and working and working - without touching F5. The script is pretty easy: $SSA = Get-SPEnterpriseSearchServiceApplication -Identity "Search Service Application"
$ContentSource = $SSA | Get-SPEnterpriseSearchCrawlContentSource -Identity "My Content Source" do {
Write-Host $ContentSource.CrawlState (Get-Date).ToString() "-" $ContentSource.SuccessCount "/" $ContentSource.WarningCount "/" $ContentSource.ErrorCount
Start-Sleep 5
} while (1) Yes, it works fine for FAST (FS4SP) Content Sources too. |
| In case of having not strong enough hardware there’s a pretty common request for start the crawl in the evening and pause in the next morning, before the work day starts. Scheduling the start of Full/Incremental Crawl is pretty easy from the admin UI, but you have to do some trick if you want to schedule the pause too. Here is my favorite trick: use PowerShell!
Here is what I do here:
- Create a script to start/resume the crawl (CrawlStart.ps1).
- Create a script to pause the crawl (CrawlPause.ps1).
- Schedule the script CrawlStart.ps1 to run in the evening (like 6pm).
- Schedule the script CrawlPause.ps1 to run in the morning (like 6am).
Is it simple, right? ;)
Here are some more details.
First, we have to know how to add the SharePoint SnapIn to PowerShell. Here is the command we need: Add-PSSnapin Microsoft.SharePoint.PowerShell.
Second, we have to get the Content Source from our Search Service Application:
$SSA = Get-SPEnterpriseSearchServiceApplication -Identity "Search Service Application"
$ContentSource = $SSA | Get-SPEnterpriseSearchCrawlContentSource -Identity "My Content Source"
Then we have to know how to check the status of this content source’s crawl: $ContentSource.CrawlStatus. Here are the available values:
- Idle
- CrawlStarting
- CrawlingIncremental / CrawlingFull
- CrawlPausing
- Paused
- CrawlResuming
- CrawlCompleting
- CrawlStopping
Finally, we have to know how to start/pause/resume the crawling:
- Start Full Crawl: $ContentSource.StartFullCrawl()
- Start Incremental Crawl: $ContentSource.StartIncrementalCrawl()
- Pause the current crawl: $ContentSource.PauseCrawl()
- Resume the crawl: $ContentSource.ResumeCrawl()
That’s it. Here are the final scripts:
1. CrawlStart.ps1
Add-PSSnapin Microsoft.SharePoint.PowerShell
$SSA = Get-SPEnterpriseSearchServiceApplication -Identity "Search Service Application"
$ContentSource = $SSA | Get-SPEnterpriseSearchCrawlContentSource -Identity "My Content Source"
if ($ContentSource.CrawlStatus -eq "Idle" ) {
$ContentSource.StartIncrementalCrawl()
Write-Host "Starting Incremental Crawl"
if ($ContentSource.CrawlStatus -eq "Paused" ) {
$ContentSource.ResumeCrawl()
Write-Host "Resuminging Incremental Crawl"
}
2. CrawlPause.ps1
Add-PSSnapin Microsoft.SharePoint.PowerShell
$SSA = Get-SPEnterpriseSearchServiceApplication -Identity "Search Service Application"
$ContentSource = $SSA | Get-SPEnterpriseSearchCrawlContentSource -Identity "My Content Source" Write-Host $ContentSource.CrawlState
if (($ContentSource.CrawlStatus -eq "CrawlingIncremental" ) -or ($ContentSource.CrawlStatus -eq "CrawlingFull" )) {
$ContentSource.PauseCrawl()
Write-Host "Pausing the current Crawl"
}
Write-host $ContentSource.CrawlState
And finally, you have to schedule these tasks as a Windows job, by using these actions: powershell –command “& ‘C:\Scripts\CrawlStart.ps1’” to start and powershell –command “& ‘C:\Scripts\CrawlPause.ps1’” to pause your crawl.
Ps.: These scripts work fine for FAST Content Sources in SharePoint too, in this case you have to use the FAST Content SSA.
Enjoy! |
| The environment: A farm with three servers: SharePoint 2010 (all rules), FAST Admin, FAST non-admin. SQL is on the SharePoint box too.
The story: Recently, I had to reinstall the box with SP2010 and SQL. Everything seemed to be fine: installing SQL, SP2010, configuring FAST Content and Query Service Apps, crawl, search… It was like a dream, almost unbelievable. But after that I started to get an error on BA Insight Longitude Connectors admin site, when I started to play with the metadata properties: “Exception configuring search settings: ... An error occurred while connecting to or communicating with the database…” I went to the FAST Query / FAST Search Administration / Managed Properties, and got this error: “Unexpected error occurred while communicating with Administration Service” Of course, I went to the SQL Server’s event log, where I found this error: “Login failed for user 'MYDOMAIN\svc_user'. Reason: Failed to open the explicitly specified database” On the Details tab I could see the ‘master’ as the related DB. I went to SQL Server Profiler, but the Trace told the same. Of course, I checked everything around FAST: the user was in the FASTSearchAdministrators group, permission settings were correct on SQL, etc. Finally, I found what I was looking for: Event Log on the FAST admin server contained this error: “System.Data.SqlClient.SqlException: Cannot open database "FASTSearchAdminDatabase" requested by the login. The login failed. Login failed for user 'MYDOMAIN\svc_user'” The solutions: Yes, it was what I was looking for: I really forgot to restore the FASTSearchAdminDatabase. But what to do if you don’t have a backup about that? Never mind, here is the Powershell command for you: Install-FASTSearchAdminDatabase -DbServer SQLServer.mydomain.local -DbName FASTSearchAdminDatabase Voilá, it’s working again! :) |
| I’m really happy because we have more and more great SharePoint events in Europe. Germany is a great example, with two major SP Conferences in this year – and I’ll be a speaker at both of them! SharePoint Connections Europe 2011 (as a part of DevConnections) 8-10 June, 2011 Karlsruhe, Germany: HAD15: Implementing Enterprise Search in SharePoint 2010
SharePoint 2010 Enterprise Search, especially with the features of the integrated FAST Search is better and more powerful than ever before. Because of the various versions and wide range of functionality, first we have to build and understand the comparison matrix of SharePoint 2010 Search. After that, I’ll demonstrate a lot of best practices for building and administering an effective search infrastructure, including SharePoint 2010 Search Engine, FAST Search Server 2010 for SharePoint, search federation, metadata management serving the effective search and information architecture investments. HAD13: Information Architecture and Enterprise Search: Better Together
Planning your Information Architecture is one of the most important factors that’s required for a successful deployment: well-organized documents and items, metadata management, Content Types, Workflows, etc. are what are you need to be familiar with. The more content you have, the more important is a well-planned and well-organized Information Architecture.
But storing the information is not enough. You also need to find the stored items and to use them in an efficient way. Enterprise Search is indispensable in this story: it is the part of your Information Architecture and completes it at the same time.
This session will demonstrate the relationship between these two as well as will demonstrate some best practices in order to help you to achieve a better and more optimal Information Architecture – integrated with Enterprise Search, all based on SharePoint. HAD12: Organizing Enterprise Documents in SharePoint 2010
In SharePoint 2010, documents can be organized not only into document libraries and folders, but also into Document Sets that provide us the capability of managing, editing and downloading documents in a set, can share metadata and version numbers, moreover we can initiate workflows for the whole document set.
This session will focus on the organizing and management of documents in document libraries, folders and document sets – including metadata management, Content Types, Content Organizer Rules, Enterprise Search, etc. European SharePoint Conference 17-20 October, Berlin, Germany: T18 - Best Practices for SharePoint 2010 Search
SharePoint 2010 Enterprise Search, especially with the features of the integrated FAST Search is better and more powerful than ever before. In my session, I will answer the following questions: • When is it enough to deploy the out-of-the-box SharePoint Search, and when do you need to install FAST Search Server as well? • When is it useful to use SharePoint 2010 as a search engine, and when you should federate? • SharePoint 2010 Search - FAST Search - Federation comparison matrix • Why is it necessary to deploy more than one Search site? • Why is it important to invest the information architecture in order to achieve and effective search architecture? • What is the connection between the metadata management and Enterprise Search? Best practices of creating a Seach-friendly taxonomy. W9 - Best Practices for Organizing Documents in SharePoint 2010 In SharePoint 2010, documents can be organized not only into document libraries and folders, but also into Document Sets that provide us the capability of managing, editing and downloading documents in a set, can share metadata and version numbers, moreover we can initiate workflows for the whole document set. In my session, I will focus on the organizing and management of documents in document libraries, folders and document sets - when to use wich one and why. I will build a decision matrix for them and help you to choose the best solution for your case. |
| Recently, I was working with FAST Search Server 2010 for SharePoint (FS4SP) and had to provide a list of all crawled properties in the category “MyCategory”. Here is my pretty simple script that provides the list in a .CSV file: $outputfile = "CrawledProperties.csv" if (Test-Path $outputfile) { clear-content $outputfile } foreach ($crawledproperty in (Get-FASTSearchMetadataCrawledProperty))
{
$category = $crawledproperty.categoryName
if ($category = "MyCategory")
{ # Get the name and type of the crawled property
$name = $crawledproperty.name
$type = $crawledproperty.VariantType switch ($type) {
20 {$typestr = "Integer"}
31 {$typestr = "Text"}
11 {$typestr = "Boolean"}
64 {$typestr = "DateTime"}
default {$typestr = “other”} }
# Build the output: $name and $typestr separated by “ “
$msg = $name + " " + $typestr Write-output $msg | out-file $outputfile -append
}
} $msg = "Crawled properties have been exported to the file " + $outputfile
write-output ""
write-output $msg
write-output ""
write-output "" |
|
|
|
|
 |
Senior Solutions Consultant,
Project Leader
at BA Insight
SharePoint Server MVP |
Email: aghy[@]aghy[.]hu
Twitter: @molnaragnes
|
|
|
|
|
|
|
|
|
|
Hosted by
|
|
|
|
|