1
0
mirror of synced 2024-07-03 02:15:17 +08:00

feat: 增加ip可用性校验,默认择快选取

This commit is contained in:
xueweihan 2022-11-22 13:11:59 +08:00
parent dea1321f44
commit 98db5e23b9
6 changed files with 113 additions and 260 deletions

179
README.md
View File

@ -1,179 +0,0 @@
# GitHub520
<p align="center">
<img src="https://cdn.jsdelivr.net/gh/521xueweihan/img_logo@main/logo/readme.gif"/>
<br><strong><a href="https://github.com/521xueweihan/HelloGitHub" target="_blank">HelloGitHub</a></strong> 分享 GitHub 上有趣、入门级的开源项目。<br>兴趣是最好的老师,这里能够帮你找到编程的兴趣!
</p>
[raw.hellogithub.com](https://raw.hellogithub.com/) 服务器续费了 3 年到 2024.12 共花了1500+💰
有余粮的朋友[点击扫码赞助](https://cdn.jsdelivr.net/gh/521xueweihan/img_logo@main/logo/receiving_code.png),感谢🙏
## 一、介绍
对 GitHub 说"爱"太难了:访问慢、图片加载不出来。
*注:* 本项目还处于测试阶段,仅在本机测试通过,如有问题欢迎提 [issues](https://github.com/521xueweihan/GitHub520/issues/new)
---
本项目无需安装任何程序,通过修改本地 hosts 文件,试图解决:
- GitHub 访问速度慢的问题
- GitHub 项目中的图片显示不出的问题
花 5 分钟时间,让你"爱"上 GitHub。
## 二、使用方法
### 2.1 复制下面的内容
```bash
# GitHub520 Host Start
140.82.114.25 alive.github.com
140.82.114.26 live.github.com
185.199.108.154 github.githubassets.com
140.82.114.22 central.github.com
185.199.108.133 desktop.githubusercontent.com
185.199.108.153 assets-cdn.github.com
185.199.108.133 camo.githubusercontent.com
185.199.108.133 github.map.fastly.net
151.101.1.194 github.global.ssl.fastly.net
140.82.114.3 gist.github.com
185.199.108.153 github.io
140.82.114.4 github.com
192.0.66.2 github.blog
140.82.114.5 api.github.com
185.199.108.133 raw.githubusercontent.com
185.199.108.133 user-images.githubusercontent.com
185.199.108.133 favicons.githubusercontent.com
185.199.108.133 avatars5.githubusercontent.com
185.199.108.133 avatars4.githubusercontent.com
185.199.108.133 avatars3.githubusercontent.com
185.199.108.133 avatars2.githubusercontent.com
185.199.108.133 avatars1.githubusercontent.com
185.199.108.133 avatars0.githubusercontent.com
185.199.108.133 avatars.githubusercontent.com
140.82.113.10 codeload.github.com
52.216.50.209 github-cloud.s3.amazonaws.com
52.217.131.145 github-com.s3.amazonaws.com
3.5.1.132 github-production-release-asset-2e65be.s3.amazonaws.com
3.5.17.197 github-production-user-asset-6210df.s3.amazonaws.com
52.217.140.201 github-production-repository-file-5c1aeb.s3.amazonaws.com
185.199.108.153 githubstatus.com
140.82.114.18 github.community
52.224.38.193 github.dev
140.82.113.22 collector.github.com
13.107.42.16 pipelines.actions.githubusercontent.com
185.199.108.133 media.githubusercontent.com
185.199.108.133 cloud.githubusercontent.com
185.199.108.133 objects.githubusercontent.com
13.107.213.51 vscode.dev
# Update time: 2022-11-22T12:05:53+08:00
# Update url: https://raw.hellogithub.com/hosts
# Star me: https://github.com/521xueweihan/GitHub520
# GitHub520 Host End
```
上面内容会自动定时更新保证最新有效。数据更新时间2022-11-22T12:05:53+08:00内容无变动不会更新
- 文件:`https://raw.hellogithub.com/hosts`
- JSON`https://raw.hellogithub.com/hosts.json`
### 2.1 手动方式
#### 2.1.1 修改 hosts 文件
hosts 文件在每个系统的位置不一,详情如下:
- Windows 系统:`C:\Windows\System32\drivers\etc\hosts`
- Linux 系统:`/etc/hosts`
- Mac苹果电脑系统`/etc/hosts`
- Android安卓系统`/system/etc/hosts`
- iPhoneiOS系统`/etc/hosts`
修改方法,把第一步的内容复制到文本末尾:
1. Windows 使用记事本。
2. Linux、Mac 使用 Root 权限:`sudo vi /etc/hosts`。
3. iPhone、iPad 须越狱、Android 必须要 root。
#### 2.1.2 激活生效
大部分情况下是直接生效,如未生效可尝试下面的办法,刷新 DNS
1. Windows在 CMD 窗口输入:`ipconfig /flushdns`
2. Linux 命令:`sudo nscd restart`,如报错则须安装:`sudo apt install nscd` 或 `sudo /etc/init.d/nscd restart`
3. Mac 命令:`sudo killall -HUP mDNSResponder`
**Tips** 上述方法无效可以尝试重启机器。
### 2.2 自动方式
**Tip**:推荐 [SwitchHosts](https://github.com/oldj/SwitchHosts) 工具管理 hosts
以 SwitchHosts 为例,看一下怎么使用的,配置参考下面:
- Title: 随意
- Type: `Remote`
- URL: `https://raw.hellogithub.com/hosts`
- Auto Refresh: 最好选 `1 hour`
如图:
![](./img/switch-hosts.png)
这样每次 hosts 有更新都能及时进行更新,免去手动更新。
### 2.3 One-liner (适用于类Unix系统)
`sed -i "/# GitHub520 Host Start/Q" /etc/hosts && curl https://raw.hellogithub.com/hosts >> /etc/hosts`
自动更新`/etc/hosts`文件可以添加到cron定时执行。使用前确保Github520内容在该文件最后部分。
### 2.4 AdGuard Home 用户(自动方式)
**过滤器>DNS 封锁清单>添加阻止列表>添加一个自定义列表**,配置如下:
- 名称: 随意
- URL: `https://raw.hellogithub.com/hosts`(和上面 SwitchHosts 使用的一样)
如图:
![](./img/AdGuard-rules.png)
更新间隔在 **设置>常规设置>过滤器更新间隔(设置一小时一次即可)**,记得勾选上 **使用过滤器和 Hosts 文件以拦截指定域名**
![](./img/AdGuard-rules2.png)
**Tip**:不要添加在 **DNS 允许清单** 内,只能添加在 **DNS 封锁清单** 才管用。另外AdGuard for Mac、AdGuard for Windows、AdGuard for Android、AdGuard for IOS 等等 **AdGuard 家族软件** 添加方法均类似。
### 2.5 Chrome 插件方式
[FasterHosts](https://github.com/gauseen/faster-hosts) 是个 Chrome 插件,主要原理是拦截浏览器的某些请求,将 `domain` 替换成访问速度较快的那个。hosts 资源来自 [GitHub520](https://github.com/521xueweihan/GitHub520),每 1 小时更新一次。
> 1. 下载 [FasterHosts](https://github.com/gauseen/faster-hosts/archive/master.zip) 然后解压,找到 `extension` 子目录
> 2. 打开 Chrome输入: `chrome://extensions/`
> 3. 打开「开发者模式」
> 4. 选择「加载已解压的扩展程序」,然后定位到刚才解压的文件夹里面的 `extension` 目录,确定
> 5. 这就安装好了,关闭「开发者模式」
## 三、效果对比
之前的样子:
![](./img/old.png)
修改完 hosts 的样子:
![](./img/new.png)
## TODO
- [x] 定时自动更新 hosts 内容
- [x] hosts 内容无变动不会更新
- [ ] 寻到最优 IP 解析结果
## 声明
<a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh"><img alt="知识共享许可协议" style="border-width: 0" src="https://licensebuttons.net/l/by-nc-nd/4.0/88x31.png"></a><br>本作品采用 <a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh">署名-非商业性使用-禁止演绎 4.0 国际</a> 进行许可。

View File

@ -126,7 +126,7 @@ hosts 文件在每个系统的位置不一,详情如下:
## TODO
- [x] 定时自动更新 hosts 内容
- [x] hosts 内容无变动不会更新
- [ ] 寻到最优 IP 解析结果
- [x] 寻到最优 IP 解析结果
## 声明

View File

@ -8,56 +8,34 @@
import os
import re
import json
import traceback
from typing import Any, Optional
from datetime import datetime, timezone, timedelta
from collections import Counter
import requests
from pythonping import ping
from requests_html import HTMLSession
from retry import retry
RAW_URL = [
"alive.github.com",
"live.github.com",
"github.githubassets.com",
"central.github.com",
"desktop.githubusercontent.com",
"assets-cdn.github.com",
"camo.githubusercontent.com",
"github.map.fastly.net",
"github.global.ssl.fastly.net",
"gist.github.com",
"github.io",
"github.com",
"github.blog",
"api.github.com",
"raw.githubusercontent.com",
"user-images.githubusercontent.com",
"favicons.githubusercontent.com",
"avatars5.githubusercontent.com",
"avatars4.githubusercontent.com",
"avatars3.githubusercontent.com",
"avatars2.githubusercontent.com",
"avatars1.githubusercontent.com",
"avatars0.githubusercontent.com",
"avatars.githubusercontent.com",
"codeload.github.com",
"github-cloud.s3.amazonaws.com",
"github-com.s3.amazonaws.com",
"github-production-release-asset-2e65be.s3.amazonaws.com",
"github-production-user-asset-6210df.s3.amazonaws.com",
"github-production-repository-file-5c1aeb.s3.amazonaws.com",
"githubstatus.com",
"github.community",
"github.dev",
"collector.github.com",
"pipelines.actions.githubusercontent.com",
"media.githubusercontent.com",
"cloud.githubusercontent.com",
"objects.githubusercontent.com",
"vscode.dev"]
IPADDRESS_PREFIX = ".ipaddress.com"
GITHUB_URLS = [
'alive.github.com', 'api.github.com', 'assets-cdn.github.com',
'avatars.githubusercontent.com', 'avatars0.githubusercontent.com',
'avatars1.githubusercontent.com', 'avatars2.githubusercontent.com',
'avatars3.githubusercontent.com', 'avatars4.githubusercontent.com',
'avatars5.githubusercontent.com', 'camo.githubusercontent.com',
'central.github.com', 'cloud.githubusercontent.com', 'codeload.github.com',
'collector.github.com', 'desktop.githubusercontent.com',
'favicons.githubusercontent.com', 'gist.github.com',
'github-cloud.s3.amazonaws.com', 'github-com.s3.amazonaws.com',
'github-production-release-asset-2e65be.s3.amazonaws.com',
'github-production-repository-file-5c1aeb.s3.amazonaws.com',
'github-production-user-asset-6210df.s3.amazonaws.com', 'github.blog',
'github.com', 'github.community', 'github.githubassets.com',
'github.global.ssl.fastly.net', 'github.io', 'github.map.fastly.net',
'githubstatus.com', 'live.github.com', 'media.githubusercontent.com',
'objects.githubusercontent.com', 'pipelines.actions.githubusercontent.com',
'raw.githubusercontent.com', 'user-images.githubusercontent.com',
'vscode.dev'
]
HOSTS_TEMPLATE = """# GitHub520 Host Start
{content}
@ -68,7 +46,7 @@ HOSTS_TEMPLATE = """# GitHub520 Host Start
# GitHub520 Host End\n"""
def write_file(hosts_content: str, update_time: str):
def write_file(hosts_content: str, update_time: str) -> bool:
output_doc_file_path = os.path.join(os.path.dirname(__file__), "README.md")
template_path = os.path.join(os.path.dirname(__file__),
"README_template.md")
@ -78,7 +56,8 @@ def write_file(hosts_content: str, update_time: str):
old_content = old_readme_fb.read()
old_hosts = old_content.split("```bash")[1].split("```")[0].strip()
old_hosts = old_hosts.split("# Update time:")[0].strip()
hosts_content_hosts = hosts_content.split("# Update time:")[0].strip()
hosts_content_hosts = hosts_content.split("# Update time:")[
0].strip()
if old_hosts == hosts_content_hosts:
print("host not change")
return False
@ -92,69 +71,82 @@ def write_file(hosts_content: str, update_time: str):
return True
def write_host_file(hosts_content: str):
def write_host_file(hosts_content: str) -> None:
output_file_path = os.path.join(os.path.dirname(__file__), 'hosts')
with open(output_file_path, "w") as output_fb:
output_fb.write(hosts_content)
def write_json_file(hosts_list: list):
def write_json_file(hosts_list: list) -> None:
output_file_path = os.path.join(os.path.dirname(__file__), 'hosts.json')
with open(output_file_path, "w") as output_fb:
json.dump(hosts_list, output_fb)
def make_ipaddress_url(raw_url: str):
"""
生成 ipaddress 对应的 url
:param raw_url: 原始 url
:return: ipaddress url
"""
return f'https://www.ipaddress.com/site/{raw_url}'
def get_best_ip(ip_list: list) -> str:
ping_timeout = 2
best_ip = ''
min_ms = ping_timeout * 1000
for ip in ip_list:
ping_result = ping(ip, timeout=ping_timeout)
if ping_result.rtt_avg_ms == ping_timeout * 1000:
# 超时认为 IP 失效
continue
else:
if ping_result.rtt_avg_ms < min_ms:
min_ms = ping_result.rtt_avg_ms
best_ip = ip
return best_ip
@retry(tries=3)
def get_ip(session: requests.session, raw_url: str):
url = make_ipaddress_url(raw_url)
def get_ip(session: Any, github_url: str) -> Optional[str]:
url = f'https://www.ipaddress.com/site/{github_url}'
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)'
' AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1'
'06.0.0.0 Safari/537.36'}
try:
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)'
' AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1'
'06.0.0.0 Safari/537.36'}
rs = session.get(url, headers=headers, timeout=5)
table = rs.html.find('.panel-item.table.table-stripes.table-v',
first=True)
pattern = r"\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b"
ip_list = re.findall(pattern, rs.text)
ip_counter_obj = Counter(ip_list).most_common(1)
if ip_counter_obj:
return raw_url, ip_counter_obj[0][0]
raise Exception("ip address empty")
ip_list = re.findall(pattern, table.text)
best_ip = get_best_ip(ip_list)
if best_ip:
return best_ip
else:
raise Exception(f"url: {github_url}, ipaddress empty")
except Exception as ex:
print("get: {}, error: {}".format(url, ex))
print(f"get: {url}, error: {ex}")
raise Exception
def main():
session = requests.session()
def main(verbose=False) -> None:
session = HTMLSession()
content = ""
content_list = []
for raw_url in RAW_URL:
for index, github_url in enumerate(GITHUB_URLS):
try:
host_name, ip = get_ip(session, raw_url)
content += ip.ljust(30) + host_name + "\n"
content_list.append((ip, host_name,))
except Exception as e:
traceback.print_exc(e)
ip = get_ip(session, github_url)
content += ip.ljust(30) + github_url + "\n"
content_list.append((ip, github_url,))
except Exception:
continue
if verbose:
print(f'process url: {index + 1}/{len(GITHUB_URLS)}')
if not content:
return
update_time = datetime.utcnow().astimezone(
timezone(timedelta(hours=8))).replace(microsecond=0).isoformat()
hosts_content = HOSTS_TEMPLATE.format(content=content, update_time=update_time)
hosts_content = HOSTS_TEMPLATE.format(content=content,
update_time=update_time)
has_change = write_file(hosts_content, update_time)
if has_change:
write_json_file(content_list)
print(hosts_content)
if verbose:
print(hosts_content)
if __name__ == '__main__':

40
hosts
View File

@ -3,6 +3,7 @@
140.82.114.26 live.github.com
185.199.108.154 github.githubassets.com
140.82.114.22 central.github.com
<<<<<<< HEAD
185.199.108.133 desktop.githubusercontent.com
185.199.108.153 assets-cdn.github.com
185.199.108.133 camo.githubusercontent.com
@ -41,6 +42,45 @@
# Update time: 2022-11-22T12:05:53+08:00
=======
185.199.109.133 desktop.githubusercontent.com
185.199.110.153 assets-cdn.github.com
185.199.109.133 camo.githubusercontent.com
185.199.109.133 github.map.fastly.net
151.101.65.194 github.global.ssl.fastly.net
140.82.114.3 gist.github.com
185.199.110.153 github.io
140.82.114.4 github.com
192.0.66.2 github.blog
140.82.114.5 api.github.com
185.199.109.133 raw.githubusercontent.com
185.199.109.133 user-images.githubusercontent.com
185.199.109.133 favicons.githubusercontent.com
185.199.109.133 avatars5.githubusercontent.com
185.199.109.133 avatars4.githubusercontent.com
185.199.109.133 avatars3.githubusercontent.com
185.199.109.133 avatars2.githubusercontent.com
185.199.109.133 avatars1.githubusercontent.com
185.199.109.133 avatars0.githubusercontent.com
185.199.109.133 avatars.githubusercontent.com
140.82.113.10 codeload.github.com
52.217.136.161 github-cloud.s3.amazonaws.com
52.217.131.145 github-com.s3.amazonaws.com
52.216.54.1 github-production-release-asset-2e65be.s3.amazonaws.com
52.217.13.236 github-production-user-asset-6210df.s3.amazonaws.com
52.217.140.201 github-production-repository-file-5c1aeb.s3.amazonaws.com
185.199.111.153 githubstatus.com
140.82.114.18 github.community
140.82.113.22 collector.github.com
13.107.42.16 pipelines.actions.githubusercontent.com
185.199.109.133 media.githubusercontent.com
185.199.109.133 cloud.githubusercontent.com
185.199.109.133 objects.githubusercontent.com
13.107.213.51 vscode.dev
# Update time: 2022-11-22T05:08:07+08:00
>>>>>>> 2161298a (feat: 增加ip可用性校验默认择快选取)
# Update url: https://raw.hellogithub.com/hosts
# Star me: https://github.com/521xueweihan/GitHub520
# GitHub520 Host End

View File

@ -1 +0,0 @@
[["140.82.114.25", "alive.github.com"], ["140.82.114.26", "live.github.com"], ["185.199.108.154", "github.githubassets.com"], ["140.82.114.22", "central.github.com"], ["185.199.108.133", "desktop.githubusercontent.com"], ["185.199.108.153", "assets-cdn.github.com"], ["185.199.108.133", "camo.githubusercontent.com"], ["185.199.108.133", "github.map.fastly.net"], ["151.101.1.194", "github.global.ssl.fastly.net"], ["140.82.114.3", "gist.github.com"], ["185.199.108.153", "github.io"], ["140.82.114.4", "github.com"], ["192.0.66.2", "github.blog"], ["140.82.114.5", "api.github.com"], ["185.199.108.133", "raw.githubusercontent.com"], ["185.199.108.133", "user-images.githubusercontent.com"], ["185.199.108.133", "favicons.githubusercontent.com"], ["185.199.108.133", "avatars5.githubusercontent.com"], ["185.199.108.133", "avatars4.githubusercontent.com"], ["185.199.108.133", "avatars3.githubusercontent.com"], ["185.199.108.133", "avatars2.githubusercontent.com"], ["185.199.108.133", "avatars1.githubusercontent.com"], ["185.199.108.133", "avatars0.githubusercontent.com"], ["185.199.108.133", "avatars.githubusercontent.com"], ["140.82.113.10", "codeload.github.com"], ["52.216.50.209", "github-cloud.s3.amazonaws.com"], ["52.217.131.145", "github-com.s3.amazonaws.com"], ["3.5.1.132", "github-production-release-asset-2e65be.s3.amazonaws.com"], ["3.5.17.197", "github-production-user-asset-6210df.s3.amazonaws.com"], ["52.217.140.201", "github-production-repository-file-5c1aeb.s3.amazonaws.com"], ["185.199.108.153", "githubstatus.com"], ["140.82.114.18", "github.community"], ["52.224.38.193", "github.dev"], ["140.82.113.22", "collector.github.com"], ["13.107.42.16", "pipelines.actions.githubusercontent.com"], ["185.199.108.133", "media.githubusercontent.com"], ["185.199.108.133", "cloud.githubusercontent.com"], ["185.199.108.133", "objects.githubusercontent.com"], ["13.107.213.51", "vscode.dev"]]

View File

@ -1,2 +1,3 @@
requests==2.23.0
requests-html==0.10.0
pythonping==1.1.4
retry==0.9.2