artificial intelligence - Smarter than an Eighth grader? Kaggle AI Challenge. R -

-

i working on allen ai science challenge on kaggle.

the idea behind challenge train model using training data provided (a set of eighth grade level science questions along 4 answer options, 1 of correct answer , correct answer) along additional knowledge sources (wikipedia, science textbooks, etc) can answer science questions (average?) eighth grader can.

i'm thinking of taking first crack @ problem in r (proficient in r , c++; don't think c++ useful language solve problem in). after exploring kaggle forums, decided use topicmodels (tm), rweka , latent dirichlet algorithm (lda) packages.

my current approach build text predictor of sort on reading question posed outputs string of text , compute cosine similarity between output text , 4 options given in test set , predict correct 1 highest cosine similarity.

i train model using training data, wikipedia corpus along few science textbooks model not overfit.

i have 2 questions here:

  1. does overall approach make sense?

  2. what starting point build text predictor? converting corpus(training data, wikipedia , textbooks) term document/document term matrix help? think forming n-grams sources don't know next step be, i.e. how model predict , belt out string of text(of say, size n) on reading question.

i have tried implementing part of approach; finding out optimum number of topics , performing lda on training set; here's code:

library(topicmodels) library(rtexttools)  data<-read.delim("cleanset.txt", header = true) data$question<-as.character(data$question) data$answera<-as.character(data$answera) data$answerb<-as.character(data$answerb) data$answerc<-as.character(data$answerc) data$answerd<-as.character(data$answerd)  matrix <- create_matrix(cbind(as.vector(data$question),as.vector(data$answera),as.vector(data$answerb),as.vector(data$answerc),as.vector(data$answerd)), language="english", removenumbers=false, stemwords=true, weighting = tm::weighttf) best.model<-lapply(seq(2,25,by=1),function(k){lda(matrix,k)}) best.model.loglik <- as.data.frame(as.matrix(lapply(best.model, loglik))) best.model.loglik.df <- data.frame(topics=c(2:25), ll=as.numeric(as.matrix(best.model.loglik))) best.model.loglik.df[which.max(best.model.loglik.df$ll),] best.model.lda<-lda(matrix,25)

any appreciated!


Comments

Post a Comment

Popular posts from this blog

php - Invalid Cofiguration - yii\base\InvalidConfigException - Yii2 -

-
Image
i've installed yii2-user module/dektrium yii2-app-basic application command composer require "dektrium/yii2-user:0.9.*@dev" config/console.php return [ . . 'modules' => [ 'gii' => 'yii\gii\module', 'user' => [ 'class' => 'dektrium\user\module', ], ], . . ] config/web.php 'components' => [ . . /* 'user' => [ 'identityclass' => 'app\models\user', 'enableautologin' => true, ], */ 'modules' => [ 'user' => [ 'class' => 'dektrium\user\module', ], ], . . ] after that, run command $ php yii migrate/up --migrationpath=@vendor/dektrium/yii2-user/migrations updating database schema . but, when run http://localhost/mylawsuit/yii/web/index.php?r=u
Read more

How to show in django cms breadcrumbs full path? -

-
need advice - how show in breadcrumbs whole path article or page? i've got code: {% ance in ancestors %} <li> {% if not forloop.last %} <a href="{{ ance.get_absolute_url }}">{{ ance.get_menu_title }}</a> {% else %} <span class="active">{{ ance.get_menu_title }}</span> {% endif %} </li> {% endfor %} but shows "home" , current page. how modify show whole path in breadcrumbs? i've found in file menu_tags.py code calls ancestors. @ least there class showbreadcrumbs: class showbreadcrumb(inclusiontag): """ shows breadcrumb node has same url current request - start level: after level should breadcrumb start? 0=home - template: template used render breadcrumb """ name = 'show_breadcrumb' template = 'menu/dummy.html' options = options( argument('start_level', default=0, require
Read more

ruby on rails - npm error: tunneling socket could not be established, cause=connect ETIMEDOUT -

-
i have app uses rails framework , implements angularjs part of front end. i have pushed heroku , have heroku toolbelt installed, when try migrate db using "heroku run rake db:migrate" receive following error(s): installing core plugins heroku-cli-addons, heroku-apps, heroku-fork, heroku-git, heroku-local, heroku-run, heroku-status... error installing package. try running again gode_debug=info see more output. ! `run` not heroku command. ! perhaps meant `-h`, `2fa`, `auth`, `join`, `open`, `orgs`, `pg`, `ps` or `rake`. ! see `heroku help` list of available commands. i run command "gode_debug=info heroku run rake db:migrate" , receive error: npm err! darwin 14.5.0 npm err! argv "/users/christopher_pelnar/.heroku/node-v4.2.1-darwin-x64/bin/node" "/users/christopher_pelnar/.heroku/node-v4.2.1-darwin-x64/lib/node_modules/npm/cli.js" "install" "heroku-cli-addons" "heroku-apps" "heroku-fork
Read more