init commit 否则项目体积会越来越大

This commit is contained in:
xueweihan 2021-01-04 09:45:04 +08:00
commit e2ac158c95
12 changed files with 620 additions and 0 deletions

51
.github/workflows/GitHub520.yml vendored Normal file
View File

@ -0,0 +1,51 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
name: GitHub520
on:
push:
schedule:
- cron: '0 */2 * * *'
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.7]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: update readme
env:
gitee_token: ${{ secrets.GITEE_TOKEN }}
gitee_gist_id: ${{ secrets.GITEE_GIST_ID }}
gitee_gist_file_name: ${{ secrets.GITEE_GIST_FILE_NAME }}
run: |
# fetch new ip content and update readme file
python fetch_ips.py
- name: commit
id: commit
run: |
git config --global user.email sunxuebangong@gmail.com
git config --global user.name action_bot
git add .
git commit -m "update readme content"
continue-on-error: true
- name: Check on failures
if: steps.commit.outputs.status == 'failure'
run: exit 1
- name: Push changes
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}

131
.gitignore vendored Normal file
View File

@ -0,0 +1,131 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
.idea/
.DS_Store

136
README.md Normal file
View File

@ -0,0 +1,136 @@
# GitHub520
<p align="center">
<img src="https://vip1.loli.net/2020/05/19/fLSBE29NxoFtOcd.gif"/>
<br><strong><a href="https://github.com/521xueweihan/HelloGitHub" target="_blank">HelloGitHub</a></strong> 分享 GitHub 上有趣、入门级的开源项目。<br>兴趣是最好的老师,这里能够帮你找到编程的兴趣!
</p>
## 一、介绍
对 GitHub 说"爱"太难了:访问慢、图片加载不出来。
*注:* 本项目还处于测试阶段,仅在本机测试通过,如有问题欢迎提 [issues](https://github.com/521xueweihan/GitHub520/issues/new)
---
本项目无需安装任何程序,通过修改本地 hosts 文件,试图解决:
- GitHub 访问速度慢的问题
- GitHub 项目中的图片显示不出的问题
花 5 分钟时间,让你"爱"上 GitHub。
## 二、使用方法
### 2.1 复制下面的内容
```bash
# GitHub520 Host Start
185.199.108.154 github.githubassets.com
199.232.96.133 camo.githubusercontent.com
199.232.96.133 github.map.fastly.net
199.232.69.194 github.global.ssl.fastly.net
140.82.112.4 gist.github.com
185.199.108.153 github.io
140.82.114.4 github.com
140.82.113.6 api.github.com
199.232.96.133 raw.githubusercontent.com
199.232.96.133 user-images.githubusercontent.com
199.232.96.133 favicons.githubusercontent.com
199.232.96.133 avatars5.githubusercontent.com
199.232.96.133 avatars4.githubusercontent.com
199.232.96.133 avatars3.githubusercontent.com
199.232.96.133 avatars2.githubusercontent.com
199.232.96.133 avatars1.githubusercontent.com
199.232.96.133 avatars0.githubusercontent.com
140.82.113.9 codeload.github.com
52.216.128.147 github-cloud.s3.amazonaws.com
52.216.109.163 github-com.s3.amazonaws.com
52.216.107.188 github-production-release-asset-2e65be.s3.amazonaws.com
52.216.207.115 github-production-user-asset-6210df.s3.amazonaws.com
52.216.146.43 github-production-repository-file-5c1aeb.s3.amazonaws.com
# Star me GitHub url: https://github.com/521xueweihan/GitHub520
# GitHub520 Host End
```
上面内容会自动定时更新保证最新有效。数据更新时间2021-01-02T14:26:43+08:00内容无变动不会更新
### 2.1 手动方式
#### 2.1.1 修改 hosts 文件
hosts 文件在每个系统的位置不一,详情如下:
- Windows 系统:`C:\Windows\System32\drivers\etc\hosts`
- Linux 系统:`/etc/hosts`
- Mac苹果电脑系统`/etc/hosts`
- Android安卓系统`/system/etc/hosts`
- iPhoneiOS系统`/etc/hosts`
修改方法,把第一步的内容复制到文本末尾:
1. Windows 使用记事本。
2. Linux、Mac 使用 Root 权限:`sudo vi /etc/hosts`。
3. iPhone、iPad 须越狱、Android 必须要 root。
#### 2.1.2 激活生效
大部分情况下是直接生效,如未生效可尝试下面的办法,刷新 DNS
1. Windows在 CMD 窗口输入:`ipconfig /flushdns`
2. Linux 命令:`sudo rcnscd restart`
3. Mac 命令:`sudo killall -HUP mDNSResponder`
**Tips** 上述方法无效可以尝试重启机器。
### 2.2 自动方式
**Tip**:推荐 [SwitchHosts](https://github.com/oldj/SwitchHosts) 工具管理 hosts
以 SwitchHosts 为例,看一下怎么使用的,配置参考下面:
- Title: 随意
- Type: `Remote`
- URL: `https://gitee.com/xueweihan/codes/6g793pm2k1hacwfbyesl464/raw?blob_name=GitHub520.yml`
- Auto Refresh: 最好选 `1 hour`
如图:
![](./img/switch-hosts.png)
这样每次 hosts 有更新都能及时进行更新,免去手动更新。
### 2.3 AdGuard Home 用户(自动方式)
**过滤器>DNS 封锁清单>添加阻止列表>添加一个自定义列表**,配置如下:
- 名称: 随意
- URL: `https://gitee.com/xueweihan/codes/6g793pm2k1hacwfbyesl464/raw?blob_name=GitHub520.yml`(和上面 SwitchHosts 使用的一样)
如图:
![](./img/AdGuard-rules.png)
更新间隔在 **设置>常规设置>过滤器更新间隔(设置一小时一次即可)**,记得勾选上 **使用过滤器和 Hosts 文件以拦截指定域名**
![](./img/AdGuard-rules2.png)
**Tip**:不要添加在 **DNS 允许清单** 内,只能添加在 **DNS 封锁清单** 才管用。另外AdGuard for Mac、AdGuard for Windows、AdGuard for Android、AdGuard for IOS 等等 **AdGuard 家族软件** 添加方法均类似。
## 三、效果对比
之前的样子:
![](./img/old.png)
修改完 hosts 的样子:
![](./img/new.png)
## TODO
- [x] 定时自动更新 hosts 内容
- [x] hosts 内容无变动不会更新
- [ ] 寻到最优 IP 解析结果
## 声明
<a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh"><img alt="知识共享许可协议" style="border-width: 0" src="https://licensebuttons.net/l/by-nc-nd/4.0/88x31.png"></a><br>本作品采用 <a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh">署名-非商业性使用-禁止演绎 4.0 国际</a> 进行许可。

110
README_template.md Normal file
View File

@ -0,0 +1,110 @@
# GitHub520
<p align="center">
<img src="https://vip1.loli.net/2020/05/19/fLSBE29NxoFtOcd.gif"/>
<br><strong><a href="https://github.com/521xueweihan/HelloGitHub" target="_blank">HelloGitHub</a></strong> 分享 GitHub 上有趣、入门级的开源项目。<br>兴趣是最好的老师,这里能够帮你找到编程的兴趣!
</p>
## 一、介绍
对 GitHub 说"爱"太难了:访问慢、图片加载不出来。
*注:* 本项目还处于测试阶段,仅在本机测试通过,如有问题欢迎提 [issues](https://github.com/521xueweihan/GitHub520/issues/new)
---
本项目无需安装任何程序,通过修改本地 hosts 文件,试图解决:
- GitHub 访问速度慢的问题
- GitHub 项目中的图片显示不出的问题
花 5 分钟时间,让你"爱"上 GitHub。
## 二、使用方法
### 2.1 复制下面的内容
```bash
{hosts_str}
```
上面内容会自动定时更新,保证最新有效。数据更新时间:{update_time}(内容无变动不会更新)
### 2.1 手动方式
#### 2.1.1 修改 hosts 文件
hosts 文件在每个系统的位置不一,详情如下:
- Windows 系统:`C:\Windows\System32\drivers\etc\hosts`
- Linux 系统:`/etc/hosts`
- Mac苹果电脑系统`/etc/hosts`
- Android安卓系统`/system/etc/hosts`
- iPhoneiOS系统`/etc/hosts`
修改方法,把第一步的内容复制到文本末尾:
1. Windows 使用记事本。
2. Linux、Mac 使用 Root 权限:`sudo vi /etc/hosts`。
3. iPhone、iPad 须越狱、Android 必须要 root。
#### 2.1.2 激活生效
大部分情况下是直接生效,如未生效可尝试下面的办法,刷新 DNS
1. Windows在 CMD 窗口输入:`ipconfig /flushdns`
2. Linux 命令:`sudo rcnscd restart`
3. Mac 命令:`sudo killall -HUP mDNSResponder`
**Tips** 上述方法无效可以尝试重启机器。
### 2.2 自动方式
**Tip**:推荐 [SwitchHosts](https://github.com/oldj/SwitchHosts) 工具管理 hosts
以 SwitchHosts 为例,看一下怎么使用的,配置参考下面:
- Title: 随意
- Type: `Remote`
- URL: `https://gitee.com/xueweihan/codes/6g793pm2k1hacwfbyesl464/raw?blob_name=GitHub520.yml`
- Auto Refresh: 最好选 `1 hour`
如图:
![](./img/switch-hosts.png)
这样每次 hosts 有更新都能及时进行更新,免去手动更新。
### 2.3 AdGuard Home 用户(自动方式)
**过滤器>DNS 封锁清单>添加阻止列表>添加一个自定义列表**,配置如下:
- 名称: 随意
- URL: `https://gitee.com/xueweihan/codes/6g793pm2k1hacwfbyesl464/raw?blob_name=GitHub520.yml`(和上面 SwitchHosts 使用的一样)
如图:
![](./img/AdGuard-rules.png)
更新间隔在 **设置>常规设置>过滤器更新间隔(设置一小时一次即可)**,记得勾选上 **使用过滤器和 Hosts 文件以拦截指定域名**
![](./img/AdGuard-rules2.png)
**Tip**:不要添加在 **DNS 允许清单** 内,只能添加在 **DNS 封锁清单** 才管用。另外AdGuard for Mac、AdGuard for Windows、AdGuard for Android、AdGuard for IOS 等等 **AdGuard 家族软件** 添加方法均类似。
## 三、效果对比
之前的样子:
![](./img/old.png)
修改完 hosts 的样子:
![](./img/new.png)
## TODO
- [x] 定时自动更新 hosts 内容
- [x] hosts 内容无变动不会更新
- [ ] 寻到最优 IP 解析结果
## 声明
<a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh"><img alt="知识共享许可协议" style="border-width: 0" src="https://licensebuttons.net/l/by-nc-nd/4.0/88x31.png"></a><br>本作品采用 <a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh">署名-非商业性使用-禁止演绎 4.0 国际</a> 进行许可。

164
fetch_ips.py Normal file
View File

@ -0,0 +1,164 @@
#!/usr/bin/env python
# -*- coding:utf-8 -*-
#
# Author : XueWeiHan
# E-mail : 595666367@qq.com
# Date : 2020-05-19 15:27
# Desc : 获取最新的 GitHub 相关域名对应 IP
import os
import re
import json
import traceback
from datetime import datetime, timezone, timedelta
from collections import Counter
from retry import retry
import requests
RAW_URL = [
"github.githubassets.com",
"camo.githubusercontent.com",
"github.map.fastly.net",
"github.global.ssl.fastly.net",
"gist.github.com",
"github.io",
"github.com",
"api.github.com",
"raw.githubusercontent.com",
"user-images.githubusercontent.com",
"favicons.githubusercontent.com",
"avatars5.githubusercontent.com",
"avatars4.githubusercontent.com",
"avatars3.githubusercontent.com",
"avatars2.githubusercontent.com",
"avatars1.githubusercontent.com",
"avatars0.githubusercontent.com",
"codeload.github.com",
"github-cloud.s3.amazonaws.com",
"github-com.s3.amazonaws.com",
"github-production-release-asset-2e65be.s3.amazonaws.com",
"github-production-user-asset-6210df.s3.amazonaws.com",
"github-production-repository-file-5c1aeb.s3.amazonaws.com"]
IPADDRESS_PREFIX = ".ipaddress.com"
HOSTS_TEMPLATE = """# GitHub520 Host Start
{content}# Star me GitHub url: https://github.com/521xueweihan/GitHub520
# GitHub520 Host End\n"""
def write_file(hosts_content: str):
update_time = datetime.utcnow().astimezone(
timezone(timedelta(hours=8))).replace(microsecond=0).isoformat()
output_doc_file_path = os.path.join(os.path.dirname(__file__), "README.md")
template_path = os.path.join(os.path.dirname(__file__),
"README_template.md")
# 应该取消 write yaml file改成 gitee gist 地址同步(国内访问流畅)
write_yaml_file(hosts_content)
with open(output_doc_file_path, "r") as old_readme_fb:
old_content = old_readme_fb.read()
old_hosts = old_content.split("```bash")[1].split("```")[0].strip()
if old_hosts == hosts_content:
print("host not change")
return False
with open(template_path, "r") as temp_fb:
template_str = temp_fb.read()
hosts_content = template_str.format(hosts_str=hosts_content,
update_time=update_time)
with open(output_doc_file_path, "w") as output_fb:
output_fb.write(hosts_content)
return True
def write_yaml_file(hosts_content: str):
output_yaml_file_path = os.path.join(os.path.dirname(__file__), 'hosts')
with open(output_yaml_file_path, "w") as output_yaml_fb:
output_yaml_fb.write(hosts_content)
def make_ipaddress_url(raw_url: str):
"""
生成 ipaddress 对应的 url
:param raw_url: 原始 url
:return: ipaddress url
"""
dot_count = raw_url.count(".")
if dot_count > 1:
raw_url_list = raw_url.split(".")
tmp_url = raw_url_list[-2] + "." + raw_url_list[-1]
ipaddress_url = "https://" + tmp_url + IPADDRESS_PREFIX + "/" + raw_url
else:
ipaddress_url = "https://" + raw_url + IPADDRESS_PREFIX
return ipaddress_url
@retry(tries=3)
def get_ip(session: requests.session, raw_url: str):
url = make_ipaddress_url(raw_url)
try:
rs = session.get(url, timeout=5)
pattern = r"\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b"
ip_list = re.findall(pattern, rs.text)
ip_counter_obj = Counter(ip_list).most_common(1)
if ip_counter_obj:
return raw_url, ip_counter_obj[0][0]
raise Exception("ip address empty")
except Exception as ex:
print("get: {}, error: {}".format(url, ex))
raise Exception
@retry(tries=3)
def update_gitee_gist(session: requests.session, host_content):
gitee_token = os.getenv("gitee_token")
gitee_gist_id = os.getenv("gitee_gist_id")
gist_file_name = os.getenv("gitee_gist_file_name")
url = "https://gitee.com/api/v5/gists/{}".format(gitee_gist_id)
headers = {
"Content-Type": "application/json"}
data = {
"access_token": gitee_token,
"files": {gist_file_name: {"content": host_content}},
"public": "true"}
json_data = json.dumps(data)
try:
response = session.patch(url, data=json_data, headers=headers,
timeout=20)
if response.status_code == 200:
print("update gitee gist success")
else:
print("update gitee gist fail: {} {}".format(response.status_code,
response.content))
except Exception as e:
traceback.print_exc(e)
raise Exception(e)
def main():
session = requests.session()
content = ""
for raw_url in RAW_URL:
try:
host_name, ip = get_ip(session, raw_url)
content += ip.ljust(30) + host_name + "\n"
except Exception:
continue
if not content:
return
hosts_content = HOSTS_TEMPLATE.format(content=content)
has_change = write_file(hosts_content)
if has_change:
try:
update_gitee_gist(session, hosts_content)
except Exception as e:
print("update gitee gist fail:{}".format(e))
print(hosts_content)
if __name__ == '__main__':
main()

26
hosts Normal file
View File

@ -0,0 +1,26 @@
# GitHub520 Host Start
185.199.108.154 github.githubassets.com
199.232.96.133 camo.githubusercontent.com
199.232.96.133 github.map.fastly.net
199.232.69.194 github.global.ssl.fastly.net
140.82.112.4 gist.github.com
185.199.108.153 github.io
140.82.114.4 github.com
140.82.113.6 api.github.com
199.232.96.133 raw.githubusercontent.com
199.232.96.133 user-images.githubusercontent.com
199.232.96.133 favicons.githubusercontent.com
199.232.96.133 avatars5.githubusercontent.com
199.232.96.133 avatars4.githubusercontent.com
199.232.96.133 avatars3.githubusercontent.com
199.232.96.133 avatars2.githubusercontent.com
199.232.96.133 avatars1.githubusercontent.com
199.232.96.133 avatars0.githubusercontent.com
140.82.113.9 codeload.github.com
52.216.128.147 github-cloud.s3.amazonaws.com
52.216.109.163 github-com.s3.amazonaws.com
52.216.107.188 github-production-release-asset-2e65be.s3.amazonaws.com
52.216.207.115 github-production-user-asset-6210df.s3.amazonaws.com
52.216.146.43 github-production-repository-file-5c1aeb.s3.amazonaws.com
# Star me GitHub url: https://github.com/521xueweihan/GitHub520
# GitHub520 Host End

BIN
img/AdGuard-rules.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 163 KiB

BIN
img/AdGuard-rules2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

BIN
img/new.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 625 KiB

BIN
img/old.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 196 KiB

BIN
img/switch-hosts.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 127 KiB

2
requirements.txt Normal file
View File

@ -0,0 +1,2 @@
requests==2.23.0
retry==0.9.2