I am convinced that the answer is that the PTFE probe is definitely safe, and it is unclear whether the PVC is safe, or whether it would leach unknown quantities of pthalates into your beer.
PTFE is the chemical name for Teflon. As we know, Teflon is considered chemically inert and safe for use in food applications, including on no-stick cookware, as long as not heated above 200°F is not applied. Many commenters have suggested a lower safe temp of below 163°F. PTFE is commonly used in other food-grade applications, such as beverage tubing, and in medical applications, such as in hoses and other medical devices (e.g., heart valves). Here is a spec sheet for a type of PTFE tubing, which shows it is chemically inert, and resistant to strong acids. Here is a data sheet from Dupont for Teflon/PTFE, stating that it is safe for food applications under FDA regs.
PVC, on the other hand, often contains plasticizers, such as phtalates (e.g., BPA) or DEHP, that leach into food products and the environment, and are considered to be hormone substitutes (they have hormonal effects on mammals). There are PVC products, such as Tygon tubing (spec sheet) that contain no pthalates and meet FDA and NSF standards for being food-safe. There are also tons of PVC products that contain pthalates. The European Union has been more active in examining the role of pthalates such as BPA in human health. Here is a link to the EU's page on this subject. They have banned BPA for use in products used by infants and children. The FDA has taken a much more laissez-faire approach to the human health issue, but they have, for example, listed DEHP (a product in PVC) as an item of concern.
Link to Wikipedia article on health effects of phtalates: link
Another consideration is that lead is often used in the manufacture of PVC coatings for electrical wires, and California's Proposition 65 requires that fact to be disclosed (link). Lead would not be used in the manufacture of PTFE.
Thus, if I had the choice, I would probably pay the extra $5 for the PTFE version.
Edit: grammar and added links.