This is just a practical approximation to the actual mathematical definition of ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jey 11 months ago \| parent \| context \| favorite \| on: A ChatGPT clone, in 3000 bytes of C, backed by GPT... This is just a practical approximation to the actual mathematical definition of GELU, which is `GELU(x) := x * Φ(x)` where Φ(x) is the CDF of the Gaussian distribution.

fluoridation 11 months ago [–]

Isn't that just erf()?

kgwgk 11 months ago | [–]

They are related but the error function approaches -1 for large negative numbers. Φ(x) approaches 0 and so does x * Φ(x).

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact