A comment on Computational Complexity blog blog inspired me to look at the following problem
Suppose you give true/false test to students and want them to put down their subjective probabilities that answer is true/false. To make them honest, you would want to award points in such a way, that in order to maximize their expected points, the student would need to report their correct probabilities. Which utility function to use? It turns out, f(x)=A log x + B is one such family. Here's the derivation
Questions:
1) Does any other such utility function exist?
2) What are other fun ways to torture students?
Here are graphs of three possible functions that can be used for eliciting probabilities from students. They are all concave, maybe one can prove any suitable function must be concave?
Yes, your idea seems to elicit correct probabilities as well. When I said f(x)=x^2 does not work, that was treating f(x) as a way of awarding points. On other hand, you are penalizing, so this equates to f(x)=-(1-x)^2 utility function which *does* achieve expected maximum at x=p
ReplyDeleteSo any utility function f(x) for which f'(x)/f'(1-x)=(1-x)/x will work. (ie expected utility EU(x) will have a local extremum at point x=p)
ReplyDeleteThis sort of looks like a differential equation...is there a good way to characterize the set of solutions?
Some smart people in comp.math commented on this, and it turns out that you can take any nice function defined on (0,1/2] and extend it to the second half by
ReplyDeletef(x) = C + int_{1/2}^x (1-t) f'(1-t)/t dt
where C is determined in a way to make the two pieces match up.
One person suggested that in order for the function to be infinitely differentiable it should look like a/x around x=1/2 ... hm, why?
http://groups-beta.google.com/group/sci.math/browse_thread/thread/767ff149b3f830ca/2589e2a14eb8dc23
The most interesting answer I got was also the simplest looking:
ReplyDeletef'(x)/f'(1-x) = (1-x)/x
rewrite that as
xf'(x) = (1-x)f'(1-x)
So basically you can see here is that the necessary condition is that
xf'(x) is some m(x) which is invariant under x->1-x
So to construct proper f(x) all we need to do is pick some m(x) that's
symmetric around x=1/2 and solve for f(x)
For instance, if m(x) = 1, then
xf'(x) = 1
f'(x) = 1/x
f(x) = log x +C
Or if m(x) = 2x(1-x), then
xf'(x) = 2x(1-x)
f'(x) = 2(1-x)
f(x) = 2x-x^2+C
f(x) = 4+2x-x^2
f(x) = (1-x)^2
Hi, I cannot figure out why "any utility function f(x) for which f'(x)/f'(1-x)=(1-x)/x will work. (ie expected utility EU(x) will have a local extremum at point x=p)".
ReplyDeleteThe link is not available now. So I cannot see your derivation. Could you post it here? Thanks a lot.
I know the derivation now. Thanks. What if the problems are multichoice ones?
ReplyDeleteBTW, Shen Yi has written a whole dissertation on issue of calibrated losses, "LOSS FUNCTIONS FOR BINARY CLASSIFICATION AND CLASS PROBABILITY ESTIMATION", page 16 has this derivation characterizing loss functions that elicit probabilities (ie, "proper scoring rules")
ReplyDeleteHemingways and their analysis about the critical side could have been really effective as they are really saying the right one. try this for the students that is very helpful for the writing services.
ReplyDeleteOur experts are proficient in thesis work. They can easily fulfil all requirement of customers and provide free service all over the world. If you wants to buy that service so click here and get remarkable online service.
ReplyDeleteWonderful post.
ReplyDeleteNice post. Probability is an important mathematical function in our high education syllabus. This is absolutely very good post about probabilities with details about the function with graphs. You can visit our site to know more about our waiver request letter writing services. This information will helpful for mathematical department students.
ReplyDeleteGraphs are best option to know the result or probabilities as you can see the clear response. Many students believe that this is their own choice to have best plan to study. https://www.litreview.net/faq-on-how-to-write-my-literature-review/ can help you a lot in many writing cases.
ReplyDeleteQuality article.
ReplyDeleteWonderful post. This is really very good article about probability with nice graphs. This methodical function and analysis should useful for students to solve there math problem. Thanks for sharing. Please check our general surgery personal statement writing services at https://residencypersonalstatements.net/our-services/general-surgery-residency-personal-statement-writing-service
ReplyDelete
ReplyDeleteExcellent machine learning blog,thanks for sharing...
Seo Internship in Bangalore
Smo Internship in Bangalore
Digital Marketing Internship Program in Bangalore
The responses are intriguing @ Yaroslav Bulatov Thank you for sharing.
ReplyDeleteSohbet
ReplyDeleteSohbet odaları
Sohbet siteleri
Sohbet sitesi
Mobil sohbet