php - Lucene Search index breaks regularly on shared hosting when site has high volume of write access -

-

i have implemented lucene on website. once every 4 days search index breaks. error saying index unreadable , site shows 500 error users.

i ssh in, rebuild index , eveything goes normal.

the part of project different normal high number of writes doing db. incrementing viewcount field on every page view. presume lucene updates document every time.

presuming issue: there way tell lucene not update index when incrementing count field?

nb: project uses sfluceneplugin within symfony

nb2: error message similar to:

sep 03 18:52:21 symfony [err] {sfexception}  file '/home/username/symfony_project/data/index/myindex/en/_1nws_s.del' not readable.  in /home/username/symfony_project/plugins/sfluceneplugin/lib/vendor/zend/search/lucene/storage/file/filesystem.php  line 59

are seeing messages in log files?

sep 03 18:52:21 symfony [err] {sfexception} file '/home/username/symfony_project/data/index/myindex/en/_1nws_s.del' not readable. in /home/username/symfony_project/plugins/sfluceneplugin/lib/vendor/zend/search/lucene/storage/file/filesystem.php line 59

if are, key point index being corrupted high number of files being open concurrently on server. limitation encountered on shared hosting, other users, if on different virtual servers, add lot of file reads/writes, webserving.

lucene creates new fragments of index each update, , on time means index spread on number of files, rather well-optimised index of 1 file. means likelihood of concurrency error increases on time badly-optimised index. optimising can help, can time-consuming large indexes , still @ risk of concurrency error, if it's lower probability.

the trick solving balance optimisation schedule using cronjob, , note, not update index trivial data changes (e.g. modified dates, view counts).

for latter point, create softupdate() method in each of model classes form part of index. create logic here discounts trivial column updates , not hook search-index updates of sfluceneplugin. now, not easy sounds sfluceneplugin uses propel behaviours run 'globally' objects...

the solution edit behaviour directly, or drop behaviour , write own methods update index. luckily, there example of functions required in symfony jobeet tutorial, day 17: http://www.symfony-project.org/jobeet/1_4/propel/en/17#chapter_17_sub_the_save_method

the downside here may end needing 'rebuild' indexing strategy you'd neatly formed in sfluceneplugin's yaml syntax in php... syntax not hard, complexity may be.

i hope makes sense, , helps in way.


Comments

Post a Comment

Popular posts from this blog

android - Spacing between the stars of a rating bar? -

-
ho can set spacing beween stars? thats ratingbar: ratingbar = (ratingbar) inflater.inflate(r.layout.ratingbar, null); ratingbar.setlayoutparams(new layoutparams(layoutparams.wrap_content, layoutparams.wrap_content)); the layout: <?xml version="1.0" encoding="utf-8"?> <ratingbar xmlns:android="http://schemas.android.com/apk/res/android" style="@style/mystyle" android:layout_alignparentright="true" android:layout_height="19dp" android:layout_width="wrap_content" android:numstars="5"> </ratingbar> the style: <style name="mystyle" parent="@android:style/widget.ratingbar"> <item name="android:progressdrawable">@drawable/android_r2_ratingstar_yellow</item> <item name="android:indeterminatedrawable">@drawable/android_r2_ratingstar_grey</item> </style> thanks, cheers you have
Read more

html - Instapaper-like algorithm -

-
does of algorithm extracts contents webpage? instapaper ? there 2 steps instapaper does: find main content block on page (excluding headers, footers, menus etc) from content block extract , format text to find content block (typically html block element, div containing key page text content) instapaper uses algorithm 1 used readability . can @ source of readability.js see what's going on, @ core tries find area on page highest text/link ratio, although has other simple scoring metrics (e.g. off top of head, things ratio of text commas, para elements etc) go heuristics. once have identified root node element, relevant content, you'll need format it, if want can pull node element containing text out of source document , insert yours, in reality you'll want remove existing styles , apply own, standard , feel. if want output nice text-only can use jericho's renderer . update1 : should mention else instapaper - follow 'pagination' links (
Read more

c# - How to execute a particular part of code asynchronously in a class -

-
i creating speech recognition engine that's going respond user commands.i have created button enable , disable speech recognition per user convenience.i have used dispose() of speech engine disable speech recoginition.here code private void button1_click(object sender, routedeventargs e) { engineon = !engineon; if (engineon) { speechengine = speech.createspeechengine(); //speech class creates , returns new speech engine. speechengine.audiolevelupdated += new eventhandler<audiolevelupdatedeventargs>(speechengine_audiolevelupdated); // use system's default microphone speechengine.setinputtodefaultaudiodevice(); speechengine.loadgrammar(new dictationgrammar()); // start listening speechengine.recognizeasync(recognizemode.multiple); } else { speechclass.myengine.dispose(); } } but disposal of speech obj
Read more