Generating matches for URL strings in elasticsearch -

-

i've been trying come right combination of tokenizers/token filters , analyzers leverage elasticsearch match urls.

unfortunately, seems every approach i've taken far misses 1 or 2 edge cases. i'm hoping out there can perhaps shed light on following:

if have following values stored in elasticsearch:

  1. http://www.example111.com
  2. http://www.example111.com/cats
  3. http://www.example111.com/cats?type=tabby
  4. http://www.example111.com/cats/dogs
  5. http://www.example111.com/dogs/cats
  6. http://www.example222.com/cats
  7. http://www.example222.com
  8. http://www.example222.com/cats/dogs
  9. http://www.example333.com/fish

i'm wondering query use generate following search string , result set combinations (ordered relevance score):

the general idea being expressed here results ranked how similar input, way down tld , scheme. results discarded when entire query string doesn't match, or segment doesn't match.

how this:

1). when store urls, url data object looks like:

{     "tld" : "http://www.example111.com",     "path" : "/cats",     "qs" : "?type=birman" }

i don't think want these analyzed... might require bit more thought.

2). when have query these records, parse url query.

3). concoct query fits requirements - so:

  • tld must match completely
  • paths in results must substring of path in query url - can use query time analyzer give possible prefix substrings of path in query url (so example: given "/cats/dogs", want "/", "/c", "/ca",..., "/cats/dogs") although seems inefficient... perhaps can pieces "/", "/cats", "/cats/dogs" beforehand when creating query , these represent additional clauses in query
  • match query string exactly? not sure full requirements here.

query might (where query url http://www.example111.com/cats/dogs?type=birman):

{     "query" : {         "bool" : {             "must" : [                 {                     "match" : {                         "url.tld" : "http://www.example111.com"                     }                 },                 {                     "match" : {                         "url.qs" : "?type=birman"                     }                 }             ]             "should" : [                 {                     "match" : {                         "url.path" : {                             "query" : "/",                             "boost" : 1                         }                     }                 },                 {                     "match" : {                         "url.path" : {                             "query" : "/cats",                             "boost" : 2                         }                     }                 },                 {                     "match" : {                         "url.path" : {                             "query" : "/cats/dogs",                             "boost" : 3                         }                     }                 }             ]         }     } }

if have multiple urls per record, nested objects , nested queries.

anyway, 1 possible idea... that's not single convenient quick query might have been hoping for.


Comments

Post a Comment

Popular posts from this blog

php - Invalid Cofiguration - yii\base\InvalidConfigException - Yii2 -

-
Image
i've installed yii2-user module/dektrium yii2-app-basic application command composer require "dektrium/yii2-user:0.9.*@dev" config/console.php return [ . . 'modules' => [ 'gii' => 'yii\gii\module', 'user' => [ 'class' => 'dektrium\user\module', ], ], . . ] config/web.php 'components' => [ . . /* 'user' => [ 'identityclass' => 'app\models\user', 'enableautologin' => true, ], */ 'modules' => [ 'user' => [ 'class' => 'dektrium\user\module', ], ], . . ] after that, run command $ php yii migrate/up --migrationpath=@vendor/dektrium/yii2-user/migrations updating database schema . but, when run http://localhost/mylawsuit/yii/web/index.php?r=u
Read more

How to show in django cms breadcrumbs full path? -

-
need advice - how show in breadcrumbs whole path article or page? i've got code: {% ance in ancestors %} <li> {% if not forloop.last %} <a href="{{ ance.get_absolute_url }}">{{ ance.get_menu_title }}</a> {% else %} <span class="active">{{ ance.get_menu_title }}</span> {% endif %} </li> {% endfor %} but shows "home" , current page. how modify show whole path in breadcrumbs? i've found in file menu_tags.py code calls ancestors. @ least there class showbreadcrumbs: class showbreadcrumb(inclusiontag): """ shows breadcrumb node has same url current request - start level: after level should breadcrumb start? 0=home - template: template used render breadcrumb """ name = 'show_breadcrumb' template = 'menu/dummy.html' options = options( argument('start_level', default=0, require
Read more

ruby on rails - npm error: tunneling socket could not be established, cause=connect ETIMEDOUT -

-
i have app uses rails framework , implements angularjs part of front end. i have pushed heroku , have heroku toolbelt installed, when try migrate db using "heroku run rake db:migrate" receive following error(s): installing core plugins heroku-cli-addons, heroku-apps, heroku-fork, heroku-git, heroku-local, heroku-run, heroku-status... error installing package. try running again gode_debug=info see more output. ! `run` not heroku command. ! perhaps meant `-h`, `2fa`, `auth`, `join`, `open`, `orgs`, `pg`, `ps` or `rake`. ! see `heroku help` list of available commands. i run command "gode_debug=info heroku run rake db:migrate" , receive error: npm err! darwin 14.5.0 npm err! argv "/users/christopher_pelnar/.heroku/node-v4.2.1-darwin-x64/bin/node" "/users/christopher_pelnar/.heroku/node-v4.2.1-darwin-x64/lib/node_modules/npm/cli.js" "install" "heroku-cli-addons" "heroku-apps" "heroku-fork
Read more